craigslist is a big believer in open source software, and relies on Linux, MySQL, Perl, Apache, Sphinx, Redis, Haraka, and many others.
craigslist (CL) has released the following open source projects:
CL blob service – a simple HTTP-based, multi-master storage service designed for scale-out and multi-datacenter deployments:
- multi-master architecture with no single point of failure
- scales well for both read and write intensive workloads
- tracks TTLs for automatic expiration and purging of blobs
- designed for both local and multiple data center replication
- pluggable index and storage interface (uses SQLite for index and filesystem for storage by default)
CL image service – large scale image resizing and processing HTTP service:
- flexible API to choose any quality, size, and center-cropping
- automatically handles orientation operations from EXIF data
- stores resized images in the blob service (could use other stores)
memcache cluster proxy (MCP) – a high-performance modular clustering HTTP reverse proxy:
- high-performance event-driven daemon
- flexible configuration, easily extensible framework
- uses standard memcached memory store for caching
- supports PCRE and rule-based traffic routing by URL, headers, and more
- tracks health of origin servers, routes around failures
- provides configurable URL redirects, rewrites, and real-time header transformations
In addition, craigslist contributes code to the following open source projects:
Haraka:
- added the smtp_client code that helps support pooled connections for proxy queues
- rewrote all of the smtp_proxy and smtp_forward code
- major contributions to the test suite
- plugin that does strict DNS checking, DNS access control lists
- contributions to bannering support
- logging infrastructure changes for more robust logging
- plugins for aliases, access lists, etc.
Redis:
- perl-AnyEvent-Redis-Federated: an event-based redis client that implements client-side sharding in Perl
- redis_util: s collection of utilities for node-redis
- provided support for included config files
Sphinx:
- provided the original persistent connections implementation
- craigslist sponsors useful sphinx feature developments, such as the TRUNCATE INDEX command for real-time sphinx
The craigslist Charitable Fund supports open source nonprofit organizations including: