A full collection happens when…

Jon Masamitsu’s Weblog : Weblog

The garbage collector tracks the average of the rate of promotion of live objects from the young generation into the tenured generation. If the average exceeds the free space in the tenured generation, the next time the young generation fills up, a full collection is done instead of a minor collection.

There are some caveats to this, read on

Performance and scalability issues doomed Friendster.

The NY Times has lengthy piece on Friendster’s business and technical problems.

Wallflower at the Web Party – New York Times

The company hired yet another chief of engineering, who laid down the law: at least 80 percent of his people would work on performance and stability issues until the Web site worked as well as it should.“In the past, we had often chosen the more exotic solution over the more simple solution,” Mr. Lindstrom said. Trailblazing a new field like social networking was enough of a challenge. “But we were also trying to innovate on the tech side as well,” he said. The company finally licked its performance problems this last summer, Mr. Lindstrom said.

On April 5th ’05, Infoworld had an article about what really was going down at Friendster.
Friendster scales the network with open source

In founding Friendster, Chairman Jonathan Abrams sought to create an online network through which friends could connect with friends. When it launched, the service was powered by a Java back end running on Apache Tomcat servers with a MySQL database. That original architecture was soon crushed by the incoming traffic.

During the summer of 2003, Friendster was plagued by performance issues. Often, the millions of users pounding the site were unable to access it, and when they could, results were inconsistent from page to page. User profile changes failed to show up because of lags in the distributed architecture, and messages were dropped.

“If you had a huge network of friends, you couldn’t search it because just building your list and comparing to the network took longer than the browser would allow you to wait,” says Dathan Pattishall, a senior database and software engineer who joined Friendster in November 2003 to tackle the site’s database issues.

Tomcat and Java weren’t the problem so much as the fact that the site’s back end was not built to accommodate millions of users. Friendster had grown so huge that simply throwing more hardware at the problem wasn’t enough. The site had to be re-engineered to make better use of the hardware and applications.

Of course, that was easier said than done. At the time, Friendster’s IT team consisted of two engineers, and the challenges they faced were daunting.

“Developing for your desktop is one thing, but when you need to support millions of hits a day, it is a different story,” Lunt says.

The Tomcat-based Java implementation was bulky, difficult to manage, and couldn’t scale to meet the surge in traffic. Moreover, the MySQL back end was a bottleneck. According to Pattishall, “Every application and sub-application had a problem. And we had to fix it.”

According to Friendster Scales-Out, Friendster Technical Environment is:

Hardware/CPU: Dual 64-bit AMD Opteron Servers
RAM: 8GB
OS: SUSE LINUX Enterprise Server 8
Database: MySQL Database Server
Language: PHP
Storage: Hitachi SAN
Load Balancer: NetScaler
Database Size: – 7.3 Terabytes
– 100s of Millions of Rows
– Over 100 Tables

True parallelism is not a feature — it’s just a remedy

David Dice’s Weblog : Weblog

Just as Java removed explicit memory management from C and C++, I expect that the next generation of programming languages may remove threads and locking as we know it today. Explicit threading will still be available, of course, but ideally I’d like to see the world switch to model where, by default, threads are managed by the runtime environment.