Skip to content
OpenParablePublic APIs

Cloud Dataproc API API

by googleapis.com

OAuthHTTPSAnalytics
Open documentation ↗

Overview

Manages Hadoop-based clusters and jobs on Google Cloud Platform.

Cloud Dataproc API is an analytics API from googleapis.com. Requests are authenticated with OAuth. It is served over HTTPS. Its OpenAPI description documents 34 endpoints.

At a glance

Authentication
OAuth
HTTPS
Yes
CORS
Unknown
Category
Analytics
Base URL
https://dataproc.googleapis.com/
Version
v1
Endpoints
34
OpenAPI
Specification ↗
Spec updated
2023-04-21

Make a first request

Replace the placeholders; the documentation says exactly where credentials go.

cURL
curl -X GET "https://dataproc.googleapis.com/v1/projects/PROJECTID/regions/REGION/clusters" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN"
JavaScript
const response = await fetch("https://dataproc.googleapis.com/v1/projects/PROJECTID/regions/REGION/clusters", {
  method: "GET",
  headers: {
    "Accept": "application/json",
    "Authorization": "Bearer YOUR_TOKEN",
  },
});
const data = await response.json();
console.log(data);
Python
import requests

response = requests.request(
    "GET",
    "https://dataproc.googleapis.com/v1/projects/PROJECTID/regions/REGION/clusters",
    headers={
    "Accept": "application/json",
    "Authorization": "Bearer YOUR_TOKEN",
    },
    timeout=30,
)
print(response.json())

Endpoints

MethodPathWhat it does
GET/v1/projects/{projectId}/regions/{region}/clustersLists all regions/{region}/clusters in a project alphabetically.
POST/v1/projects/{projectId}/regions/{region}/clustersCreates a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cl
GET/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}Gets the resource representation for a cluster in a project.
PATCH/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}Updates a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cl
DELETE/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}Deletes a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cl
POST/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:diagnoseGets cluster diagnostic information. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/goog
POST/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:repairRepairs a cluster.
POST/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:startStarts a cluster in a project.
POST/v1/projects/{projectId}/regions/{region}/clusters/{clusterName}:stopStops a cluster in a project.
GET/v1/projects/{projectId}/regions/{region}/jobsLists regions/{region}/jobs in a project.
GET/v1/projects/{projectId}/regions/{region}/jobs/{jobId}Gets the resource representation for a job in a project.
PATCH/v1/projects/{projectId}/regions/{region}/jobs/{jobId}Updates a job in a project.
DELETE/v1/projects/{projectId}/regions/{region}/jobs/{jobId}Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.
POST/v1/projects/{projectId}/regions/{region}/jobs/{jobId}:cancelStarts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list (https://cloud.google.com/dataproc/docs/refere
POST/v1/projects/{projectId}/regions/{region}/jobs:submitSubmits a job to a cluster.
POST/v1/projects/{projectId}/regions/{region}/jobs:submitAsOperationSubmits job to a cluster.
GET/v1/{name}Retrieves the latest workflow template.Can retrieve previously instantiated template by specifying optional version parameter.
PUT/v1/{name}Updates (replaces) workflow template. The updated template must contain version that matches the current server version.
DELETE/v1/{name}Deletes a workflow template. It does not cancel in-progress workflows.
POST/v1/{name}:cancelStarts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. If the serv
POST/v1/{name}:instantiateInstantiates a template and begins execution.The returned Operation can be used to track execution of workflow by polling operations.get. The Operation will com
POST/v1/{name}:resizeResizes a node group in a cluster. The returned Operation.metadata is NodeGroupOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cl
GET/v1/{parent}/autoscalingPoliciesLists autoscaling policies in the project.
POST/v1/{parent}/autoscalingPoliciesCreates new autoscaling policy.
GET/v1/{parent}/batchesLists batch workloads.
POST/v1/{parent}/batchesCreates a batch workload that executes asynchronously.
POST/v1/{parent}/nodeGroupsCreates a node group in a cluster. The returned Operation.metadata is NodeGroupOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cl
GET/v1/{parent}/workflowTemplatesLists workflows that match the specified filter in the request.
POST/v1/{parent}/workflowTemplatesCreates new workflow template.
POST/v1/{parent}/workflowTemplates:instantiateInlineInstantiates a template and begins execution.This method is equivalent to executing the sequence CreateWorkflowTemplate, InstantiateWorkflowTemplate, DeleteWork
POST/v1/{project}/{region}/{cluster}:injectCredentialsInject encrypted credentials into all of the VMs in a cluster.The target cluster must be a personal auth cluster assigned to the user who is issuing the RPC.
POST/v1/{resource}:getIamPolicyGets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.
POST/v1/{resource}:setIamPolicySets the access control policy on the specified resource. Replaces any existing policy.Can return NOT_FOUND, INVALID_ARGUMENT, and PERMISSION_DENIED errors.
POST/v1/{resource}:testIamPermissionsReturns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a NOT_FOUND e

Questions

Does the Cloud Dataproc API API need an API key?

Yes. The Cloud Dataproc API API uses OAuth authentication; the documentation explains how to get credentials.

Does the Cloud Dataproc API API support HTTPS?

Yes, the Cloud Dataproc API API is served over HTTPS.

How many endpoints does the Cloud Dataproc API API have?

Its OpenAPI description lists 34 endpoints in version v1. All of them are shown on this page.

What are alternatives to the Cloud Dataproc API API?

Other Analytics APIs include Google Play Integrity API, versionhistory.googleapis.com API, Doc Converter, Perspective Comment Analyzer API, Safe Browsing API.

Compare alternatives →

Google Play Integrity API

Analytics

The Play Integrity API helps you check that you're interacting with your genuine app on a genuine Android device powered by Google Play services. The Play Integrity API has replaced SafetyNet Attestation and Android Device Verification.

OAuth1 endpoints

versionhistory.googleapis.com API

Analytics

Version History API - Prod

No auth4 endpoints

Doc Converter

Analytics

This api converts file formats of OpenXml and OpenOffice documents formats to vector files (e.g., svg)

No auth1 endpoints

Perspective Comment Analyzer API

Analytics

The Perspective Comment Analyzer API provides information about the potential impact of a comment on a conversation (e.g. it can provide a score for the "toxicity" of a comment). Users can leverage the "SuggestCommentScore" method to submit corrections to improve Perspective over time. Users can set the "doNotStore" flag to ensure that all submitted comments are automatically deleted after scores are returned.

OAuth2 endpoints

Safe Browsing API

Analytics

Enables client applications to check web resources (most commonly URLs) against Google-generated lists of unsafe web resources. The Safe Browsing APIs are for non-commercial use only. If you need to use APIs to detect malicious URLs for commercial purposes – meaning “for sale or revenue-generating purposes” – please refer to the Web Risk API.

No auth7 endpoints

Web Fonts Developer API

Analytics

The Google Web Fonts Developer API lets you retrieve information about web fonts served by Google.

No auth1 endpoints

Listed from APIs.guru (CC0).