- (Optional) Request a Test Google Account
- Verify if already installed by typing
gcloudin bash terminal - Follow the appropriate quickstart for your OS
- Initialize tool
gcloud init- Create your own project when prompted
- Set default application credentials
gcloud auth application-default login
For Java Developers:
- Download/Install IntelliJ Community Edition
- Download/Install OpenJDK 8 or Java JDK 8
For Python Developers:
For Developers with only a browser:
- Clone GitHub repos in cloud shell and use code editor
-
Shorcut to automatically clone DataflowTemplates
- Alternatively in cloud shell run:
git clone https://github.com/GoogleCloudPlatform/DataflowTemplates.git -
Shortcut to automatically clone beam-workshop
- Alternatively in cloud shell run:
git clone https://github.com/danieldeleo/beam-workshop.git
-
On the welcome screen:
- Select Checkout from Version Control
- Choose Git from the dropdown
Enter git repo URL:
- For Java Developers enter:
- For Python Developers enter:
Set the following required pipeline arguments:
- --runner=DataflowRunner
- --project=YOUR_PROJECT_ID
More Dataflow Pipeline Options
Launch IntelliJ and on the welcome screen:
- Select Checkout from Version Control
- Choose Git from the dropdown
- Enter git repo URL: https://github.com/apache/beam.git
The following commands should be run in the sdks/python directory
virtualenv envsource env/bin/activatepip install -e .[gcp]