Server for 100 users

GoSnaps: 500, 000 users in 5 days on $100/month server

Similarly to GoChat, I also launched a Pokémon GO fan app last week, called GoSnaps. GoSnaps is an app to share Pokémon GO screenshots and images on a map. The Instagram/Snapchat for Pokémon GO. GoSnaps grew to 60k users its first day, 160k users on its second day and 500k unique users after 5 days (which is now). It has 150–200k uploaded snaps now. It has around 1000 concurrent users at any given time. I built image recognition software to automatically check if an uploaded image is Pokémon GO-related, and resizing tools for uploaded images. We run this whole setup on one medium Google Cloud server of $100/month, plus (cheap) Google Cloud Storage for the storage of images. Yes, $100. And it performs well.

Technical comparison of GoChat and GoSnaps

Let’s compare GoChat and GoSnaps. Both apps probably fire a lot of requests per second to fetch chats/images within a certain area of the map. This is a geospatial lookup in the database (or search engine), either by a polygon of latitude/longitude locations or by a specific point. We use a polygon and we fire this request every time someone moves the map. These types of queries are heavy operations on a database, especially in combination with sorting or filtering. We get this type of search request hundreds of times per second. GoChat probably did too.

Unique to GoChat is that it had to fetch and post a lot of chat messages every second. The article about GoChat talks about 600 requests per second for the whole app. Those 600 requests are a combination of map requests and chat messages. These chat messages are small and could/should be done over a simple socket connection, but happen often and have to be distributed to other chatters. This is manageable with the right setup, but disastrous with a poor, MVP-like setup.

GoSnaps, on the other hand, has a lot of images being fetched and ‘liked’ every second. The snaps pile up on the server, since old snaps stay relevant. Old chats do not. Since the actual image files are stored in the Google Cloud Storage, the amount of requested image files is not a concern for me as a developer. Google Cloud handles this and I trust Google. But the requested snaps on the map are my concern. GoSnaps has image recognition software that looks for patterns on all uploads to see if an image is Pokémon-related or not. It also resizes the images and sends them to Cloud Storage. These are all heavy operations in terms of CPU and bandwidth. Way heavier than distributing some small chat messages, but less frequent.

My conclusion is that both apps are very similar in terms of scalability complexity. GoChat handles more small messages while GoSnaps handles larger images and heavier server operations. Designing an architecture for these two apps both require a slightly different approach, but are similarly complex.

How I built a scalable MVP in 24h

GoSnaps is built as an MVP, not as a professional business product. It was built entirely in 24 hours. I took a NodeJS boilerplate project for hackathons and used a MongoDB database without any form of caching. No Redis, no Varnish, no fancy Nginx settings, nothing. The actual iOS app was built in native Objective-C code, with some borrowed Apple Maps-related code from Unboxd, our main app. So how did I make it scalable? By not being lazy.

Let’s say I would consider an MVP as solely a race against the clock to build a functional app as quick as possible, regardless of technical backend quality. Where would I have put my images? In the database: MongoDB. It would require no configuration and almost no code. Easy. MVP. How would I have queried the snaps within a certain area that got the most likes? By just running a plain MongoDB query on the entire pile of uploaded snaps. Just one database query on one database collection. MVP. All of this would have destroyed my app and the app’s feature.

Look at the query I would have had to run to get these snaps: “find all snaps within location polygon [A, B, C, D], excluding snaps marked as abuse, excluding snaps that are still being processed, ordered by number of likes, ordered by valid Pokémon GO snaps first and then ordered by newest first”. This works great on a small dataset, great, MVP. But this would have been totally disastrous under any type of serious load. Even if I would have simplified the above query to only include three conditions/sorting operations, it would have been disastrous. Why? Because this is not how a database is supposed to be used. A database should query only on one index at a time, which is impossible with these geospatial queries. You’ll get away with it if you don’t have a lot of users, but you’ll go down once you get successful. Like GoChat.

What did I do instead? After applying the CPU-expensive image recognition and doing resizing, the resized images are uploaded to Google Cloud Storage. This way the server and database don’t get hit for requesting images. The database should worry about data, not images. This saves many servers by itself. On the database side, I separate the snaps into a few different collections: all snaps, most liked snaps, newest snaps, newest valid snaps and so forth. Whenever a snap gets added, liked or marked as abuse, the code checks if it (still) belongs to one of those collections and acts accordingly. This way the code can query from prepared collections instead of running complicated queries on one huge pile of mess. It’s simply separating data logically into some simple buckets. Nothing complicated. But it allows me to query solely on the geospatial coordinates with one sorting operation, instead of a complex query as described above. In simple terms: it makes it straightforward to select data.

Cimsun(CHINA) Tech Co.,Ltd CIMSUN CimFAX H5 Fax from Desktop High Speed Fax Server 100 Users Fax from Office/Home/on Field Fax2Email Professional Fax System For Windows
CE (Cimsun(CHINA) Tech Co.,Ltd)
  • Send/receive faxes from PC (desktop) to fax machines / fax servers / online fax; Faxes queue up to send according to priority;
  • Fax to email; Get fax on mobile devices; Auto save fax as PDF/TIFF files; Manually save fax as PDF, image, or TIF; Get fax by email on mobile devices
  • Fax doc, pdf, excel, image and documents the like; Fax from software/document/scanner; Sign/Stamp electronically;
  • Sound scheme for sending and receiving fax; Auto resend when fax failed; Faxing status synchronized; Fax in high resolution;
  • Supports Windows xp/7/8/10; 100 users licensed; 1 CimFAX100 fax machines; 4 easy steps to set up; 4GB memory stores up to 80, pages; software-and-hardware integrated;
Essentially KateS Peppermint Essential Oil 4 oz. with Detailed User's Guide E-book and Glass Dropper by Essentially KateS.
Home (Essentially KateS)
  • Each bottle comes with a long glass dropper, a phenolic cap for long-term storage, and our detailed User s Guide, sent by e-mail attachment when your peppermint...
  • Peppermint oil is a natural repellent of mice and other rodents, spiders, mosquitos, and other insects. Simple how-to instructions are included in the User s Guide.
  • Peppermint oil is anti-bacterial, anti-viral, and anti-fungal, which makes it an excellent natural all purpose cleaner to help keep your home and kitchen clean...
  • If you are unhappy with our product for any reason, we will provide a full refund or replacement, at your request. No explanation necessary. Your satisfaction is...
  • Essentially KateS Premium Essential Oils currently include Peppermint, Tea Tree, Lavender, Frankincense, and Ylang Ylang.
STARTECH.COM StarTech USB 3.0 to Gigabit Ethernet NIC Network Adapter - USB to RJ45 for 10/100/1000 Network
Personal Computer (STARTECH.COM)
  • Compatible with Gigabit Ethernet networks and powered via USB for versatility and portability as an add-on or replacement
  • Uses built-in Checksum and Large Send Offload support and Jumbo Frames, the Gigabit USB adapter helps conserve system resources
  • Backed by a StarTech 2-year warranty and free lifetime technical support
  • USB to RJ45 adapter supports Gigabit networking over USB 3.0
  • I 802.3, 802.3u and 802.3ab (10BASE-T, 100BASE-TX, and 1BASE-T) compatible
  • Crossover Detection & Auto-Correction (Auto MDIX)
  • I 802.1Q VLAN Tagging support
  • Powered directly from the USB ports

Related posts: