API/CLI Reference

#API/CLI Reference

#API-Reference

#burla.remote_parallel_map

Run a Python function on many remote computers at the same time.

remote_parallel_map(
  function_,
  inputs,
  func_cpu=1,
  func_ram=4,
  func_gpu=None,
  image=None,
  grow=False,
  max_parallelism=None,
  detach=False,
  generator=False,
  spinner=True
)

Run provided function_ on each item in inputs in at the same time using all available workers.
Extra inputs are queued and processed sequentially on each worker.

If grow=True automatically boots and assigns additional workers to minimize runtime.

While running:

• If the provided function_ raises an exception, the exception is raised on the client machine.
• Your print statements (anything written to stdout/stderr) are streamed back to your local machine, appearing like they would have if running the same code locally.

When finished remote_parallel_map returns a list of objects returned by each function_ call.
Optionally, it can return a generator that yields results as they become available.

#CLI-Reference

Burla's CLI contains the following commands:

• burla install Deploy self-hosted Burla in your Google Cloud project.
• burla login Connect your computer to the cluster you last logged into in the browser.

The global arg --help can be placed after any command or command group to see CLI documentation.

#burla install

Deploy a self-hosted Burla instance in your current Google Cloud Project.
Running burla install multiple times will update the existing installation with the latest version.

Description:

Installs Burla inside the Google Cloud project that your gcloud CLI is currently pointing to.
For a more user-friendly installation guide see: Installation: Self-Hosted

To view your current gcloud project run: gcloud config get project
To change your current gcloud project run: gcloud config set project <desired-project-id>

Prerequisites:

• Have the gcloud CLI installed (how do I install the gcloud CLI?).
• Be logged in to the gcloud CLI (how do I log in?)
  (gcloud auth login & gcloud auth application-default login)
• Have a Google Cloud user account with the minimum required permissions to install Burla.
  Or: Just run burla install, if you're missing any permissions it will tell you which ones!

Here is the set of permissions you'll need to run burla install:
Any of these three permission set's will work.

PROJECT_ID="$(gcloud config get-value project)"
gcloud iam roles create burlaInstaller \
  --project "$PROJECT_ID" \
  --file burla-installer-role.yaml

title: "Burla Installer"
description: "Minimum permissions required to install Burla"
stage: "GA"
includedPermissions:
  # Enable required APIs
  - serviceusage.services.enable
  - serviceusage.services.get
  - serviceusage.services.list

  # Read project number + set project IAM bindings
  - resourcemanager.projects.get
  - resourcemanager.projects.getIamPolicy
  - resourcemanager.projects.setIamPolicy

  # Firewall rule for cluster nodes
  - compute.firewalls.create
  - compute.firewalls.get
  - compute.firewalls.list

  # Create + manage bucket
  - storage.buckets.create
  - storage.buckets.get
  - storage.buckets.list
  - storage.buckets.update

  # Secret for cluster token + IAM bindings + versions
  - secretmanager.secrets.create
  - secretmanager.secrets.get
  - secretmanager.secrets.list
  - secretmanager.secrets.getIamPolicy
  - secretmanager.secrets.setIamPolicy
  - secretmanager.versions.add
  - secretmanager.versions.access

  # Create service accounts, grant them roles, rotate keys, and use them for Cloud Run deploy
  - iam.serviceAccounts.create
  - iam.serviceAccounts.get
  - iam.serviceAccounts.list
  - iam.serviceAccounts.getIamPolicy
  - iam.serviceAccounts.setIamPolicy
  - iam.serviceAccounts.actAs
  - iam.serviceAccountKeys.create
  - iam.serviceAccountKeys.list
  - iam.serviceAccountKeys.delete

  # Create Firestore database + write initial config doc via Firestore client
  - datastore.databases.create
  - datastore.databases.get
  - datastore.entities.create
  - datastore.entities.update
  - datastore.entities.get

  # Deploy and configure Cloud Run service
  - run.services.create
  - run.services.get
  - run.services.list
  - run.services.update
  - run.services.getIamPolicy
  - run.services.setIamPolicy

These permissions are only required to install Burla, they are not granted to the Burla service itself.
Upon install Burla creates a service account for itself having this minimum set of permissions.

We encourage you to check out _install.py in the client for even more specific installation details.

#burla login

Connects your computer to the Burla cluster you most recently logged into in your browser.
Authorizes your machine to call remote_parallel_map on this cluster.

Description:

Launches the "Authorize this Machine" page in your default web browser.

If there is no auth-cookie (you have not yet logged into the dashboard), throws simple error requesting you login to your cluster dashboard first.

When the "Authorize" button is hit, a new auth token is created and sent to your machine.

This token is saved in the text file burla_credentials.json. This file is stored in your operating system's recommended user data directory which is determined using the appdirs python library.

This token is refreshed each time the burla login is run, or certain amount of time passes.

Questions?
Schedule a call with us, or email jake@burla.dev. We're always happy to talk.