Software Engineering Points

I have tried to distill some knowledge that has come from building a few green field projects. These are the three points that have come about from this experience.

• Focus on building up a language of "higher concepts" that are relevant to your system.
• As your system progresses into later stages of development the hope is that you will be able to begin to spend more time applying the "higher concepts" you built in earlier stages.
• Naming, Modularization, and Code Organization should not be under estimated.
• Giving modules a well defined purpose and exporting a public interface is crucial. While it may not affect how the the code works it allows you to think at a higher level and keep less information in your brain at once. KISS
• The eventual goal is to build software that is "product ready", in other words you want to be able to release the software and leave it alone.
• If you have to hire a big IT team to support your software maybe you can improve your code to be more self sufficient.

IaaS vs. PaaS vs. SaaS

Saas, A simple example of SaaS is an online email service, like Gmail. SaaS is really geared toward the end users in your organization and doesn't take much to get started.

IaaS is at the other end of the cloud spectrum. In this scenario, you want to maintain control of your software environment, but you don't want to maintain any equipment. You don't want to have to buy servers and put them in a climate-controlled room or any of that. Instead, you go to an IaaS provider and request a virtual machine.

PaaS is somewhere in between IaaS and SaaS. It's not a finished product, like SaaS, and it's not a tabula rasa, like IaaS. PaaS gives your application developers hooks and tools to develop to that particular platform. For example, Microsoft's Windows Azure gives you tools to develop mobile apps, social apps, websites, games and more. You build these things, but you use the APIs and tools to hook them into the Azure environment and run them there.

Decoupling your architecture

Messaging Techniques

Spring Remoting
Remote Method Invocation (RMI)
Spring's HTTP invoker
Hessian
Burlap
JAX RPC
JMS

RabbitMQ
ZeroMQ

REST

Naming Servers
doozer

Abstraction on top of messaging
Camel abstracts away your messaging layer so you can just call methods on services and if you route everything properly.
Spring Integration also does this but we had issues with the way it considers a service only a method while we wanted an
entire class to represent a service with multiple methods

Serialization Protocol
Java Serialization only works in java land and not in a polyglot system.
Protocol Buffers seems to fit nicely here

Doozer

Doozer is a network service. A handful of machines (usually three, five, or seven) each run one doozer server process. These processes communicate with each other using a standard fully-consistent distributed consensus algorithm. Clients dial in to one or more of the doozer servers, issue commands, such as GET, SET, and WATCH, and receive responses.

Each doozerd process has a complete copy of the datastore and serves both read and write requests; there is no distinguished "master" or "leader". Doozer is designed to store data that fits entirely in memory; it never writes data to permanent files. A separate tool provides durable storage for backup and recovery.

https://github.com/ha/doozerd

Meebo talk on asynchronous architecture

Watched the below post on architecture over at Meebo

Got me thinking about:

Pre Forking with aynchronous servers

In general I was interested in the issue of running asynchronous servers on multiple processors/cores.

Seems that this can be done in multiple servers:

Ruby Unicorn
Python Gunicorn
Facebook Tornado Python server


Meebo talk on asynchronous architecture

Apache CGI process per request - memory efficiency sucks!
Apache mod_wsgi
Twisted
Gevent + Gunicorn

http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-using-coroutines-to-create-efficient-high-concurrency-web-applications-4900984

-----

Gevent + Gunicorn

For SSL support, both Gunicorn and Tornado recommend you run them behind a reverse proxy such as nginx.

Gunicorn - Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model ported from Ruby's Unicorn project.

Greins - Greins is a Gunicorn application which makes it easy to configure and manage any number of WSGI apps in one server daemon.

https://github.com/meebo/greins

https://github.com/srlindsay/gevent-profiler

http://stackoverflow.com/questions/13852752/is-gevent-gunicorn-scalable-and-stable-for-production-use

------

Facebook Tornado

 Ways to run Tornado

  1. listen: simple single-process:

  2. bind/start: simple multi-process:

  3. add_sockets: advanced multi-process:

http://www.tornadoweb.org/documentation/httpserver.html

 Background
The "pre-forking" phrase is used to mean a couple of different things, so it is worth clarifying what it means for Tornado. Essentially, we bind a single server socket to a port, and we fork() one process per CPU. Each of those CPUs calls accept() on the shared server socket, and the Linux kernel gives new requests to one of the child processes on a first-come-first-serve basis. Each of the child processes has their own epoll-based IO loop, and a single request is handled entirely within one of the child processes. There is no shared memory or shared state between the forked child processes, only a shared port.

-----

In the classic setup you have nginx sending requests to a pool of mongrels using a smart balancer or a simple round robin. Eventually you want better visibility and reliability out of your load balancing situation, so you throw haproxy into the mix:

Http Proxy     Load Balancer       Cluster of web servers
Nginx      ->   Haproxy        -> (Mongrel or Unicorn or …)


https://github.com/blog/517-unicorn

-----

Ruby's unicorn

 Unicorn is a traditional UNIX prefork web server. No threads are used at all, this makes applications easier to debug and fix. When your application goes awry, a BOFH can just “kill -9” the runaway worker process without worrying about tearing all clients down, just one. Only UNIX-like systems supporting fork() and file descriptor inheritance are supported.

http://unicorn.bogomips.org/DESIGN.html


While reading up on these topics I found out about WSGI which is the equivalent of Ruby's Rack as far as I can tell.

http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface

Quick note on how nginx fits in.

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites. In contrast to other HTTP accelerators, such as Squid, which began life as a client-side cache, or Apache and nginx, which are BOLDprimarily origin serversBOLD, Varnish was designed as an HTTP accelerator. Varnish is focused exclusively on HTTP, unlike other proxy servers that often support FTP, SMTP and other network protocols.

Considerations in Building a Large Infrastructure

Analyze your types of servers for instance Facebook had 5 types of servers:
Web page servers, database computers, data storage systems, news feed servers and something called memcache servers

Possible infrastructure I could imagine being implemented

DNS           Load balancers          Web servers                    Databases and Data Storage
round  --> 1 --------------->      group of web servers                   Session Storage instead of sticky sessions on a web server?
robin            2 --------------->      2nd group of web servers               Databases SQL or NoSQL distributed for failover and also size of data
with Finally Batch Processing and Analytics
Health Check

Good article on load balancing

DNS roundrobin is excellent to increase capacity, by distributing the load across multiple points (potentially geographically distributed). But it does not provide fail-over. You must first describe what type of failure you are trying to cover. A server failure must be covered locally using a standard IP address takeover mechanism (VRRP, CARP, ...). A switch failure is covered by resilient links on the server to two switches. A WAN link failure can be covered by a multi-link setup between you and your provider, using either a routing protocol or a layer2 solution (eg: multi-link PPP). A site failure should be covered by BGP : your IP addresses are replicated over multiple sites and you announce them to the net only where they are available.

http://serverfault.com/questions/101053/is-round-robin-dns-good-enough-for-load-balancing-static-content
http://stackoverflow.com/questions/1472214/load-balancing-dns-round-robin-in-front-of-hardware-load-balancers-how-to-shar