Every public Airbnb listing in Inside Airbnb's open data dump, 1.4M photos we could pull from public listing pages, and every one of 50M reviews. CLIP, YOLOv8, and Claude Haiku running in parallel on Burla, in 10.8 hours of wall time.
Each dot is a listing flagged by one of the photo detectors below, color-coded by category. Drag, zoom, click for the listing.
YOLOv8 confirmed a TV in the upper half of the photo and CLIP also flagged it as “TV mounted above a fireplace.” Click any photo for the full image and the listing.
Top by CLIP score against “a messy cluttered room with stuff everywhere.” Click a photo to expand it. We are not naming the listings; the link is there if you really need to.
CLIP “room full of houseplants” combined with YOLOv8 potted-plant counts.
A 3-tier funnel: regex on every review, embedding cluster on the top 200k, Claude Haiku on the top 10k. Click any card to read the full review.
Burla ships a Python function plus its imports to a fleet of
containers running the standard python:3.12 image. The
orchestrator returns them as a single iterator. The shared filesystem
at ./shared is GCSFuse-mounted to a GCS bucket and
visible from every worker, which is what makes this kind of
slice-and-merge pipeline cheap.
At peak, Burla had 1,000 containers cooperating on the photo pipeline and another 1,000 chewing through 50M reviews. The whole 1.4M-photo index moved through CLIP in roughly 24 minutes once the fleet was warm.
Full writeup is on GitHub. Burla docs are at docs.burla.dev.
from burla import remote_parallel_map def score_batch(args: ScoreArgs) -> list[dict]: import torch, open_clip, pyarrow.parquet as pq model, _, prep = open_clip.create_model_and_transforms( "ViT-B-32", pretrained="laion2b_s34b_b79k", cache_dir="./shared/airbnb/clip_weights", ) # read row group, encode, score against prompts, # write a parquet shard back to /workspace/shared. results = remote_parallel_map( score_batch, batch_args, # ~6,000 row-group batches func_cpu=2, func_ram=8, max_parallelism=1000, # 1k concurrent at peak grow=True, spinner=True, )
For each idea below, we sort all 1.1M listings into a few groups (like “darkest photos” vs “brightest photos”) and check whether the more popular ones really do land in one group. Two of four ideas held up. Two did not.