On July 11, 2026, a site called Lobsters did something that sounds counterintuitive: they swapped the paid database system they’d run for over a decade (a commercial package called MariaDB that needed its own server to run) for a completely free database — SQLite. You can think of the latter as a “file-based” database: no separate server, no extra bill, runs with a few lines of code.
Two days later on Monday morning, one of the site’s maintainers posted internally: CPU usage dropped, memory footprint dropped, page loads got smoother. The key line: “Once that MariaDB server is fully decommissioned, the monthly VPS bill is cut in half.”
In programmer circles, the post exploded. 384 upvotes, 92 comments. Not because the tech was flashy — quite the opposite: because it was so plainly modest.
Seven Years of Database Agony
Before talking about this migration, a quick word on what Lobsters is. It’s a “link-sharing + discussion” site for programmers, a quieter, more hardcore version of Hacker News. Users share tech articles; others vote and comment. The site isn’t large — the database file is about 500MB, and ordinary daily traffic fits on a regular server — but it’s run stably for over a decade.
The problem was its database. Years ago, Lobsters chose MariaDB — a commercial database system requiring a standalone server. Over time, the team came to feel the setup was too heavy: one more server meant one more monthly bill and one more potential failure point to maintain. In August 2018, lead maintainer pushcx opened a GitHub discussion titled “Discuss migrating to PostgreSQL” — another paid database.
The discussion thread lay dormant for seven years, its direction drifting from PostgreSQL to SQLite. The real turning point came in early 2025: investment group K1 acquired MariaDB, raising community doubts about its long-term prospects. Around the same time, a community member named Rahul asked in the thread: “Can Lobsters run on SQLite?”
What is SQLite? In one sentence: a free database that stores all its data in a local file. No installation, no configuration, no separate server. It’s embedded in Chrome, in WeChat, in every app on your phone — the most widely deployed database engine in the world. But for a long time it was widely considered “unsuitable for websites,” because its design differs in philosophy from traditional server-requiring database systems (MariaDB, PostgreSQL, MySQL).
Rahul’s question overturned that default assumption.
First Migration: CPU Pegged at 100%, Emergency Rollback
In June 2025, a community contributor named thomas0 formally took on the migration. He recorded a rare, candid account in his post: the whole process went through three code-submission attempts, one failed launch, and then three fixes before finally succeeding.
The first launch happened on February 21, 2026. thomas0 and pushcx got on a call and drew up a detailed deployment checklist; everything proceeded as planned — until the moment the new code went live. The site entered read-only mode (a protective measure to prevent data corruption), but merely handling users’ browsing requests pegged all server CPU at 100%. Frozen. The two debugged for ages but couldn’t find the cause. The decision: roll back.
thomas0 wrote in his post: “I didn’t feel great after that failure.” Because he’d known beforehand that, lacking access to the production database, performance issues were a hidden risk — and his guess was confirmed.
Post-mortem, the problem was in three places. Two of them were SQLite performing “full table scans” on the two largest tables in the database — like finding a book in a library by flipping through every shelf from the first row, ignoring the catalog’s index numbers. With small data it’s fine; with large data, every time someone opens a page the server has to read the entire table from start to finish, maxing out the CPU. The third was an inefficient pattern called “N+1 queries”: for each row fetched, the program fired N extra queries. The correct approach is to pull all needed data in one go.
Three problems — two in the SQL wording, one in the program logic. None were flaws in SQLite itself — it’s that identical code produces wildly different execution efficiency between two different database systems.
Second Migration: A Quiet Monday Morning
After the February 21 rollback, thomas0 submitted the third fix in just two days. What did he do?
First, he fixed the two full-table-scan problems found on first launch: he added proper indexes to the queries — essentially building a fast-lookup catalog for those “big tables.” For anyone familiar with databases, this is basic; but for a migration scenario, the key point is: on MariaDB, these queries may have taken a different execution path, so the performance problem never surfaced. After switching to SQLite, the same query statement, SQLite chose a different execution strategy — a full scan. Change the database, and yesterday’s “good code” becomes “bad code.”
Second, he fixed the N+1 queries: changed the looped queries into batch queries. The program no longer asks the database one row at a time, but pulls all needed data at once.
Third, he spent a week using a script he wrote to generate locally half of Lobsters’ real data volume as test data — because he couldn’t get the real production data, he had to simulate traffic this way. The script itself was extra engineering effort.
Fourth, for safety, he added a “slow query log” switch before launch: if any undiscovered performance issue remained, the system would automatically log queries taking over 100 milliseconds, making quick pinpointing easy.
On July 11, 2026, the second launch. This time the ending was different. The site stayed up normally; CPU and memory curves were smooth. They watched user feedback in the chat channel, handled two minor issues, then waited for Monday — the real test of peak traffic.
Monday morning, all calm. pushcx said in internal chat: “We had a quiet Monday.”
Why Did a “Simpler” Database Turn Out Better?
The counterintuitive part of this story is here: SQLite is far more “bare-bones” than MariaDB — it has no user-permission system, doesn’t support heavy concurrent writes, can’t be accessed remotely over a network, and doesn’t support many advanced query syntaxes. Yet after Lobsters switched to it, everything got better.
There are three layers of reasons.
Layer one: one fewer server, one fewer pile of headaches. In the old architecture, Lobsters’ web app ran on one server and the MariaDB database on another. The two machines needed network communication, and separate maintenance, backups, and monitoring. SQLite turned the database into a file inside the web app — data lives on the same machine, and backup is just copying a file. For a site like Lobsters that “one server can handle all its traffic,” a standalone database server isn’t an asset, it’s a liability.
Layer two: kill the latency. Every time a user opens a page, the web app needs to query the database. In the MariaDB architecture, that query goes “app → network → database server → network → app,” a round trip. After switching to SQLite, the query becomes “app → local file,” and the network-latency variable is eliminated entirely. For read-heavy, write-light sites — like a link-sharing site — this change delivers a real, tangible speed boost.
Layer three: cost. This is the most direct. The monthly rent for that MariaDB server is no longer paid. The VPS bill is halved. This isn’t some abstract “cost reduction and efficiency gain”; it’s one fewer number on the bill.
thomas0 also listed some technical details in his post: SQLite doesn’t support unsigned big integers, so certain ID field types had to change; SQLite’s collation is weaker than MariaDB’s, supporting only ASCII case-insensitivity, not full UTF-8 collation handling; he used user-defined functions to patch a few computation features SQLite lacks. These details don’t matter to ordinary readers, but they illustrate a principle: the essence of migration is finding, between two systems, a new set of paths that let all functionality keep working.
”Good Enough” vs. the Software Industry’s “Complexity Worship”
The reason this story deserves a wider audience isn’t technical. It touches a deep-rooted habit in the software industry: defaulting to the “big and comprehensive” solution rather than the “good enough” one.
Lobsters originally chose MariaDB because the standard practice for building websites back then was “app + standalone database server.” That architecture was reasonable a decade-plus ago — back then growth expectations were high, traffic swings were large, and the database needed buffering capacity. But a decade-plus later, Lobsters’ scale hasn’t qualitatively changed. It’s still a mid-sized community site whose daily traffic a regular server can absorb. Yet that “just in case” database server kept generating a fixed monthly expense.
This isn’t isolated. In the software industry, there’s a common error called “premature optimization”: paying upfront for a scale that hasn’t arrived. A three-person startup team stands up a Kubernetes cluster, microservices architecture, and master-slave databases — all for “future scalability.” These choices aren’t wrong in themselves, but the price is a triple increase in operational complexity, monthly bills, and troubleshooting difficulty.
A layer deeper, what I want to point out is: technical “advanced” and “appropriate” are two different things. A free, file-storing lightweight database really is less flashy on the feature list than a commercial one. But if you don’t need those extra features — multi-user permissions, geo-replication, massive concurrent writes — then those features aren’t assets, they’re baggage.
Of course, this doesn’t mean SQLite fits every scenario. thomas0 himself admitted in the comment-section discussion that if a site has heavy concurrent-write needs, requires multiple servers accessing the same data simultaneously, or needs complex user-permission management, SQLite isn’t the right choice. Its concurrency model is “multiple readers, single writer” — many people reading at once is fine, but only one person can write at any given moment. For a community like Lobsters where “users browse far more than they post,” this isn’t a problem. For Taobao or WeChat, it would be a disaster.
The key is hidden in the act of examining what you actually need. That judgment matters more than which database version number you pick.
Finally
From August 2018, when pushcx opened that discussion thread, to July 11, 2026, when the second launch succeeded, Lobsters’ database migration spanned nearly eight years. In between: one failure, three code revisions, a self-written test script, a self-written database-migration tool, and countless discussions and “should we try again?” messages in the chat channel.
The final result is simple enough to state in one sentence: a long-running veteran tech community swapped its database from a commercial system requiring a standalone server to a free, file-based one. The server bill halved. Monday morning was quiet.
This isn’t a story about “disruption.” It’s a story about “returning to good enough.”
Reference Links
- Lobsters internal post: Now running on SQLite (posted by thomas0)
- GitHub issue #539: the complete history of migrating from MariaDB to PostgreSQL/SQLite
- Simon Willison’s report: Lobsters has migrated to SQLite
- pushcx’s deployment checklist Gist: the complete steps for both launches
- Lobsters open-source code repository (GitHub)