Automating 10,000+ Products Across 30 Stores: Architecture Notes

The work that does not scale by hand

A dropshipper's day job is repetitive and enormous: pull products from a source, rewrite listings, set prices with a margin, push them to a marketplace, then keep all of that in sync as source prices and stock move. One store is tedious. Thirty stores with thousands of products each is impossible to do by hand, and the manual version is where the mistakes live.

Why a queue, not a loop

The naive version is a big loop: for each store, for each product, call the marketplace API. It works in a demo and dies in production. Marketplaces rate-limit you, calls fail intermittently, and a single loop means one slow store blocks every other store behind it.

So the core is a job queue. Every unit of work — sync one product, reprice one listing, refresh one store's stats — is a job. Workers pull jobs and run them concurrently, with retries and backoff baked in. The dashboard just enqueues intent; the workers do the slow, failure-prone talking to the outside world.

• Each store's work runs independently, so a slow or rate-limited store never blocks the others.
• Failed jobs retry with backoff instead of aborting a whole run.
• Throughput scales by adding workers, not by rewriting the logic.
• The dashboard stays instant because it never waits on a marketplace API — it just drops jobs on the queue.

The stack

NuxtJS for the dashboard, Prisma over PostgreSQL for state, Python workers for the heavy integration jobs, all wrapped in Docker so the whole thing runs the same on my machine as in production. Nothing exotic — the leverage comes from the shape of the system, not the brand names in it.

Days of manual product management collapsed into minutes. Not because the code was clever, but because the work stopped being sequential.

What I would tell anyone building marketplace automation

1. 1Treat every external API call as something that will fail, and design the retry before the happy path.
2. 2Isolate tenants (stores) so one bad actor's rate limit is not everyone's outage.
3. 3Keep the UI off the critical path — enqueue intent, report status, never block a click on a third-party API.