DockerCon EU 2015 Barcelona

November 29, 2015

I’m sharing with you my impressions on one of the best conferences I’ve ever attended.
It is one of the best, due to the technology leap that Docker is introducing now and will introduce in the future.
Cloud is impossible without containers.

Advertisements

Repair Canon EF 16-35mm F/2.8L II USM

January 6, 2015

In a recent month I visited Canada and it was very cold, lower than -30C. I took many pictures with my Canon 16-35mm F/2.8L 2 usm, it is my favorite lens. In some day when I was taking pictures, I had noticed that the lens is not sitting firmly in its mount, but it appeared that the lens was broken some how and was wobbling. I searched the internet and found the following blog page http://psy-factor.livejournal.com/752711.html showing the broken lens part and explaining how to disassemble the lens and replace the part. I had ordered a part from ebay and replaced the part.

Many thanks to the owner of the blog http://psy-factor.livejournal.com/ Vladimir, he was kindly helping me with the disassembly and answered my questions. Thanks a lot Vladimir.

Here are disassembly pictures:

2015-01-03 15.03.27 2015-01-03 15.03.33 2015-01-03 15.03.46 2015-01-03 15.04.00 2015-01-03 15.04.07 2015-01-03 15.04.49 2015-01-03 15.07.49 2015-01-03 15.07.52 2015-01-03 15.13.19 2015-01-03 15.14.39 2015-01-03 15.14.44 2015-01-03 15.15.19 2015-01-03 15.18.16 2015-01-03 15.19.33 2015-01-03 15.25.46 2015-01-03 15.27.11 2015-01-03 15.45.48 2015-01-03 15.45.56 2015-01-03 15.55.25 2015-01-03 16.26.26 2015-01-03 16.26.30 2015-01-03 16.32.37 2015-01-03 16.32.40 2015-01-03 16.36.15 2015-01-03 16.36.22 2015-01-03 16.42.09 2015-01-03 16.42.17 2015-01-03 16.42.20 2015-01-03 17.01.36 2015-01-03 17.21.39 2015-01-03 17.53.07 2015-01-03 17.53.31

This was the broken part which I replaced.

I think the plastic handles were broken due to big temperature differences (about 55C degrees). Probably Canon has used cheap plastic parts on high-end lenses. Shame on you Canon!

2015-01-03 17.54.06

Performance Tests on different versions of Jackson Scala Module 2_10 libraries

November 11, 2014

Recently I performed performance tests on one of our server side components and saw very significant impact on performance. After some investigations I have managed to isolate the culprit of the performance degradation, it is the Jackson library (for json parsing), here are the results:

In my tests I use some relatively big json string ~11KBytes.
The test performs deserialization from json to object 10000 times, repeats it 10 times and makes an average.

def measure(repeat : Int, times : Int) = {
    var start = 0L
    var end = 0L
    var diff = 0L
    var sum = 0L
    (1 to repeat).foreach { rep =>
      start = System.currentTimeMillis()
      (1 to times).par.foreach { i =>
        val hh = JsonLayer.JSON_MAPPER.readValue(householdString("hh" + i * rep), classOf[Household])
        //val st = JsonLayer.JSON_MAPPER.writeValueAsString(hh)
      }
      end = System.currentTimeMillis()
      diff = (end - start)
      sum += diff
      logger.debug("Round " + rep + " took " + diff + " millis")
    }
    sum
  }

  @Test
  def householdJsonPerformance() =
  {
    val repeat = 10
    val times = 10000
    var start = 0L
    var end = 0L
    var sum  = 0L
    var diff = 0L

    // warmup
    diff = measure(repeat, times)
    logger.debug("Round warm up took " + diff + " millis")

    Measure.time(this.logger, "performance household deserialize/serialize") {
          sum = measure(repeat, times)
      }
    logger.debug("Average per run " + (sum/repeat) + " millis")
  }

Jackson libraries version: 2.4.3

jackson-core-2.4.3.jar
jackson-annotations-2.4.3.jar
jackson-databind-2.4.3.jar

The only difference between the tests is following jar were updated each time:

jackson-module-scala_2.10-{version}.jar

The results of the test above with different versions of the jars:

2.2.2

Round 1 took 2921 millis
Round 2 took 842 millis
Round 3 took 677 millis
Round 4 took 597 millis
Round 5 took 556 millis
Round 6 took 590 millis
Round 7 took 697 millis
Round 8 took 597 millis
Round 9 took 577 millis
Round 10 took 548 millis
Round warm up took 8602 millis
[ Executing performance household deserialize/serialize ...]
Round 1 took 474 millis
Round 2 took 458 millis
Round 3 took 465 millis
Round 4 took 493 millis
Round 5 took 519 millis
Round 6 took 445 millis
Round 7 took 446 millis
Round 8 took 485 millis
Round 9 took 485 millis
Round 10 took 456 millis
[ performance household deserialize/serialize took 4731 ms ]
Average per run 472 millis millis

2.2.3

Round 1 took 8228 millis
Round 2 took 5428 millis
Round 3 took 3771 millis
Round 4 took 3798 millis
Round 5 took 3560 millis
Round 6 took 3558 millis
Round 7 took 3847 millis
Round 8 took 3709 millis
Round 9 took 3496 millis
Round 10 took 3836 millis
Round warm up took 43231 millis
[ Executing performance household deserialize/serialize ...]
Round 1 took 5355 millis
Round 2 took 6018 millis
Round 3 took 3962 millis
Round 4 took 3481 millis
Round 5 took 3821 millis
Round 6 took 3739 millis
Round 7 took 3545 millis
Round 8 took 3594 millis
Round 9 took 3615 millis
Round 10 took 3595 millis
[ performance household deserialize/serialize took 40726 ms ]
Average per run 4072 millis

2.3.3


Round 1 took 8631 millis
Round 2 took 5342 millis
Round 3 took 3617 millis
Round 4 took 3578 millis
Round 5 took 3633 millis
Round 6 took 3740 millis
Round 7 took 3541 millis
Round 8 took 3833 millis
Round 9 took 3678 millis
Round 10 took 3596 millis
Round warm up took 43189 millis
[ Executing performance household deserialize/serialize ...]
Round 1 took 3753 millis
Round 2 took 3408 millis
Round 3 took 4728 millis
Round 4 took 4270 millis
Round 5 took 3565 millis
Round 6 took 3521 millis
Round 7 took 3492 millis
Round 8 took 3573 millis
Round 9 took 3959 millis
Round 10 took 4094 millis
[ performance household deserialize/serialize took 38365 ms ]
Average per run 3836 millis

2.4.3

Round 1 took 8331 millis
Round 2 took 4903 millis
Round 3 took 3568 millis
Round 4 took 3545 millis
Round 5 took 3494 millis
Round 6 took 3597 millis
Round 7 took 3637 millis
Round 8 took 3629 millis
Round 9 took 3612 millis
Round 10 took 3550 millis
Round warm up took 41866 millis
[ Executing performance household deserialize/serialize ...]
Round 1 took 3611 millis
Round 2 took 3510 millis
Round 3 took 3508 millis
Round 4 took 3923 millis
Round 5 took 4848 millis
Round 6 took 4819 millis
Round 7 took 3960 millis
Round 8 took 3580 millis
Round 9 took 4092 millis
Round 10 took 4480 millis
[ performance household deserialize/serialize took 40332 ms ]
Average per run 4033 millis

To summarize:

There is performance degradation in jackson scala module 2_10 libraries starting from 2.2.3 version (2.2.2 version works fine)

In my tests above version jackson-module-scala_2.10-2.2.2 work ~7.5 times faster that jackson-module-scala_2.10-2.2.3

Jackson module scala 2_10 version Average latency
2.2.2 ——————————– 472 ms
2.2.3 ——————————– 4072 ms
2.3.3 ——————————– 3836 ms
2.4.3 ——————————– 4033 ms

There is an open ticket for that issue since May 2014

https://github.com/FasterXML/jackson-module-scala/issues/148

Spring 3 – SPEL Expression

September 17, 2014

To use the configuration parameter in SPEL expressions use the following syntax:

<property name="someProperty" value="#{ '${config.parameter}' == 'some.string.to.compare.to' ? false : true }" />

Socket: Connection Refused – No Timeout

May 18, 2014

Some one has asked me: Your server failed connecting to third party server, it retries and immediately gets a response: Connection Refused. Why is not there a time-out?

Here is capture from a log:

DEBUG:  Creating a new socket to '10.38.6.62:1119'...
ERROR:  Failed reconnecting: Connection refused
DEBUG:  Creating a new socket to '10.38.6.62:1119'...
ERROR:  Error - Connection refused [ socket = null ] Going to retry for the '3' time.
DEBUG:  Creating a new socket to '10.38.6.62:1119'...
ERROR:  Failed reconnecting: Connection refused
ERROR:  Could not communicate with 10.38.6.62:1119. '3' retries are ended. Total request time '11' ms

The answer is:

Connection Refused means that there is no program at the other end of the connection that listens to incoming connections. Connection time-outs are for the cases where there is no response at all. A connection refusal is a valid response, so the time-out doesn’t apply in such case.

Spray vs Scalatra

May 15, 2014

During last few evenings I have evaluated two of the well known frameworks:

Spray http://spray.io/

  • REST/HTTP framework developed in Scala on top of Akka

AND

Scalatra – http://www.scalatra.org/

  • Web application framework developed in Scala
  • It is a port of Sinatra framework written in Ruby

————————————————————————-

My goal was creating few simple web pages to receive (GET/POST) requests, go to third-party http server for results and return response to the client.

I have started from Spray framework:

The main class looks like this:

It creates Http server and connects it with Akka

Akka IO will receive all the communication and redirect it to Spray’s Http layer, which in his turn redirects it to service actor of type HttpRoutingService shown below

import spray.can.Http
import akka.actor.{Props, ActorRef, ActorSystem}
import akka.io.IO

/**
 * Main Class
 *
 */
object Main extends App {

  try {
    // we need an ActorSystem to host our application in
    implicit val system = ActorSystem("on-spray-can")

    // create and start our service actor
    val service = system.actorOf(Props[HttpRoutingService], "routing-service")

    // start a new HTTP server on port 8080 with our service actor as the handler
    IO(Http) ! Http.Bind(service, "localhost", port = 8080)

  }
  catch
    {
      case t : Throwable => println(t)
    }
}

Let’s look at HttpRoutingService class

import scala.concurrent.duration._
import akka.pattern.ask
import akka.util.Timeout
import akka.actor._
import spray.can.Http
import spray.can.server.Stats
import spray.util._
import spray.http._
import HttpMethods._
import MediaTypes._
import spray.can.Http.RegisterChunkHandler

import java.io.File
import org.parboiled.common.FileUtils
import scala.concurrent.duration._
import akka.actor._
import akka.pattern.ask
import spray.routing.{HttpService, RequestContext}
import spray.routing.directives.CachingDirectives
import spray.can.server.Stats
import spray.can.Http
import spray.httpx.marshalling.Marshaller
import spray.httpx.encoding.Gzip
import spray.util._
import spray.http._
import MediaTypes._
import CachingDirectives._

// we don't implement our route structure directly in the service actor because
// we want to be able to test it independently, without having to spin up an actor
class HttpRoutingService extends Actor with RoutingService {

  // the HttpService trait defines only one abstract member, which
  // connects the services environment to the enclosing actor or test
  def actorRefFactory = context


  // this actor only runs our route, but you could add
  // other things here, like request stream processing,
  // timeout handling or alternative handler registration
  def receive = runRoute(httpRoutes)
}


// this trait defines our service behavior independently from the service actor
trait RoutingService extends HttpService {

  // we use the enclosing ActorContext's or ActorSystem's dispatcher for our Futures and Scheduler
  implicit def executionContext = actorRefFactory.dispatcher

  val loginActor = ActorSystem().actorOf(Props[LoginActor])

  val httpRoutes =
    get
    {
      pathSingleSlash {
          complete(index)
      } ~
      path("login") {
          complete(loginForm)
      }
    } ~
    post {
      path("loginForm") {
        //entity(as[Login]){ login =>
        entity(as[FormData]) { fields =>

          println("Fields = " + fields)
          val username = fields.fields(0)._2
          val password = fields.fields(1)._2
          val login = Login(username, password)
          println("Login = " + login)
          loginActor! login
          complete("Done")
        }
      }
    }


  lazy val index =
    <html>
      <body>
        <h1>Say hello to <i>spray-routing</i> on <i>spray-can</i>!</h1>
        <p>Defined resources:</p>
        <ul>
          <li><a href="/login">/login</a></li>
        </ul>
      </body>
    </html>


  lazy val loginForm =
    <div class="login_block">
      <form name="LoginForm" action="/loginForm"  method="post">
        <table class="login">
          <tbody>
            <tr>
            <td>
              <input type="text" name="username" id="login_email" />
            </td>
            </tr>
            <tr>
            <td>
               <input type="text" name="password" id="Irealpass" />
            </td>
            </tr>
            <tr>
               <td align="left"><input type="submit" name="login" /></td>
            </tr>
          </tbody>
        </table>
      </form>
    </div>
 
}

The class defines few Http routes:

  • HTTP GET / -> serves the default page (index page)
  • HTTP GET /login -> shows a simple submit form
  • HTTP POST /loginForm -> receives a submit form, takes out form fields and redirects request to other actor

I would like to say that I have spent a lot of time to make this routing service to work. I had trouble compiling the class, due to some errors with tildas and problems with getting out the fields from a form (this was a nightmare… I do not know why it should so complex) The syntax is not easy nor intuitive for understanding at all (at least for me). There are some tildas… I cannot understand why do we need them?! We don’t have them in Scalatra.

Here is the code of additional actor – LoginActor
This actor receives a login message with username and password, sends http request for login to some third-party server.

It was important to me to get the hands on the spray http client and try it out. It also was not so straight forward working with it. I had some trouble with HttpHeaders and its huge hierarchy…
Anyway, I find the dispatch http client the easiest of all to use.
Dispatch Http Client is a Scala wrapper for Java Async Http Client

import akka.actor.{ActorSystem, ActorLogging, Actor}
import akka.util.Timeout
import java.util.concurrent.TimeUnit
import spray.http._
import scala.concurrent.Future
import akka.io.IO
import spray.can._
import spray.client.pipelining._
import HttpMethods._
import spray.http.HttpRequest
import spray.http.HttpResponse
import spray.http.HttpHeaders.{RawHeader, Accept}
import scala.util.{Success, Failure}
import spray.httpx.marshalling.Marshaller


/**
 * Created by roma on 5/13/14.
 */
class LoginActor extends Actor with ActorLogging {

  implicit val timeout: Timeout = Timeout(15, TimeUnit.SECONDS)
  implicit val system: ActorSystem = ActorSystem()
  import system.dispatcher // implicit execution context


  def receive() =
  {
     case Login(username, password) => login(username, password)
  }

     
  def login(username : String, password : String) =
  {
     log.info("Logging in with Username = " + username + " ...")
     val headers =  List[HttpHeader](
                    RawHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"),
                    RawHeader("Origin", "http://www.server.com"),
                    RawHeader("Content-Type", "application/x-www-form-urlencoded"),
                    RawHeader("Referer", "http://www.server.com/"),
                    RawHeader("Accept-Encoding", "gzip,deflate,sdch"),
                    RawHeader("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6,he;q=0.4,ru;q=0.2"))



     val formData = FormData(List(("Username",username), ("Password", password)))
     val entity = HttpEntity.NonEmpty(ContentType(MediaTypes.`application/x-www-form-urlencoded`), HttpData(formData))
     val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
     val response: Future[HttpResponse] = pipeline(Get("http://www.server.com/login.php").withHeaders(headers).withEntity(formData))
      response onComplete{
        case Failure(ex) => log.error("Failed logging in - " + ex.getMessage)
        case Success(resp) =>
          resp.status.intValue match
          {
            case 200 =>
              log.debug("Body: \n" + resp.entity.asString)
              log.info("Successfully logged in as '" + username + "'")
            case _  =>
              log.error("Failed logging in - " + resp.status)
          }
      }

  }

}

Scalatra

After Spray I found Scalatra very easy to use and work with. Intuitive APIs without any complex syntax (remember tildas in Spray?)

Scalatra requires a specific project structure (actually the same as all other web projects):

Scalatra's Project Structure

I just copied the webapp directory from one of the Scalatra’s examples.
In my set up I used embedded jetty as Http serlvet container for Scalatra, here is how it is done:

The main class is a JettyLauncher class:

A simple class which starts up a Jetty Server and passes it Scalatra’s listener servlet


import akka.actor.{Props, ActorRef, ActorSystem}
import akka.io.IO
import org.eclipse.jetty.servlet.DefaultServlet
import org.scalatra.servlet.ScalatraListener
import org.eclipse.jetty.webapp.WebAppContext
import org.eclipse.jetty.server.Server
import com.typesafe.scalalogging.slf4j.LazyLogging

/**
 * Main Class
 *
 */
object JettyLauncher extends App with LazyLogging {

  try {

    logger.info("Starting up ...")

    val port = if(System.getenv("PORT") != null) System.getenv("PORT").toInt else 8080

    logger.info("Using port " + port)

    val server = new Server(port)
    val context = new WebAppContext()
    context setContextPath "/"
    context.setResourceBase("src/main/webapp")
    context.addEventListener(new ScalatraListener)
    context.addServlet(classOf[DefaultServlet], "/")

    server.setHandler(context)

    server.start

    logger.info("Running.")

    server.join

  }
  catch
    {
      case t : Throwable => logger.error("ERROR", t)
    }
}

ScalatraBootstrap class bind Scalatra’s Listener with the ScalatraServlet. In our case it’s called HttpHandlerService

import org.scalatra._
import javax.servlet.ServletContext

class ScalatraBootstrap extends LifeCycle {
  override def init(context: ServletContext) {
    context.mount(new HttpHandlerService, "/*")
  }
}
import akka.util.Timeout
import akka.actor._
import akka.pattern.ask
import org.scalatra.ScalatraServlet
import com.typesafe.scalalogging.slf4j.LazyLogging
import java.util.concurrent.TimeUnit
import dispatch.Defaults.executor
import scala.concurrent.{Future, Await}
import scala.concurrent.duration._
import com.ning.http.client.Response

class HttpHandlerService extends ScalatraServlet with LazyLogging {

  implicit val timeout = Timeout(10, TimeUnit.SECONDS)

  logger.debug("Creating actors ...")

  val loginActor= ActorSystem().actorOf(Props[LoginActor])

  logger.debug("done.")


  get("/")
  {
    index
  }

  get("/login") {
      <html><body>{loginForm}</body></html>
  }

  post("/loginForm") {

        val username = params("username")
        val password = params("password")
        val login = Login(username, password)
        logger.debug("Sending to loginActor...")
        val futureData = loginActor.ask(login)
        logger.debug("Done. Waiting for response ...")

        val responseData = futureData map { data =>  }
        logger.debug("Received response.")
        responseData match
        {
          case ResponseData(response) => logger.debug(response.getResponseBody); <html><body>"Logged in as " {username}</body></html>
          case other @ _ => logger.debug("Other data type: " + other); <html><body>"Other Data Type received: " { other }</body></html>
        }
  }



  lazy val index =
    <html>
      <body>
        <h1>Welcome to <i>bouncer</i> on <i>scalatra</i>!</h1>
        <p>Defined resources:</p>
        <ul>
          <li><a href="/login">Login</a></li>
        </ul>
      </body>
    </html>


  
  lazy val loginForm =
          <section class="loginform cf">
          <form name="login" action="index_submit" method="get" accept-charset="utf-8">
            <ul>
              <li>
                <label for="usermail">Email</label>
                <input type="email" name="usermail" placeholder="yourname@email.com" required="true" />
              </li>
              <li>
                <label for="password">Password</label>
                <input type="password" name="password" placeholder="password" required="true" />
              </li>
              <li>
                <input type="submit" value="login" />
              </li>
              </ul>
            </form>
          </section>


  
}

I really liked the way the Scalatra defines routes. Very easy and clear, without additional noisy declarations. It took me 10 minutes to start the Scalatra’s service. Well Done Scalatra guys!

Here is the LoginActor class with the usage of Dispatch http client:

import akka.actor.{ActorRef, ActorSystem, ActorLogging, Actor}
import akka.util.Timeout
import java.util.concurrent.{ExecutorService, Executors, TimeUnit}
import scala.util.{Random, Success, Failure}
import dispatch._
import com.typesafe.scalalogging.slf4j.LazyLogging
import com.ning.http.client._
import com.ning.http.client.AsyncHandler.STATE
import com.ning.http.client.cookie.Cookie
import scala.collection.JavaConversions._

/**
 * Created by roma on 5/13/14.
 */
class LoginActor extends Actor with LazyLogging {

  implicit val timeout: Timeout = Timeout(15, TimeUnit.SECONDS)
  implicit val system: ActorSystem = ActorSystem()
  import system.dispatcher // implicit execution context

  private var isRunning : Boolean = false
  private var lastRun : Long = 0

  def receive() =
  {
     case Login(username, password) => sender ! Data(login(username, password))
  }

  
  def login(username : String, password : String) =
  {
     logger.info("Logging in with Username = " + username + " ...")

     // create request
     val request = url("""http://www.server.com/login.php""").POST

     // add headers
      request.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
      request.addHeader("Origin", "http://www.yad2.co.il")
      request.addHeader("Content-Type", "application/x-www-form-urlencoded")
      request.addHeader("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36")
      request.addHeader("Referer", "http://www.server.com/")
      request.addHeader("Accept-Encoding", "gzip,deflate,sdch")
      request.addHeader("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6,he;q=0.4,ru;q=0.2")

      request.addParameter("Username", username)
      request.addParameter("Password", password)

      val futureResponse = Http(request OK as.String)

      futureResponse

  }

}

I really liked how the Dispatch framework works. Recommend you to try it out – link

I used maven so here are the maven dependencies:

      <dependency>
          <groupId>org.scalatra</groupId>
          <artifactId>scalatra_2.10</artifactId>
          <version>2.2.0</version>
      </dependency>


      <dependency>
          <groupId>com.typesafe.akka</groupId>
          <artifactId>akka-actor_2.10</artifactId>
          <version>2.3.2</version>
      </dependency>

      <dependency>
          <groupId>org.scalatra.scalate</groupId>
          <artifactId>scalate-web_2.10</artifactId>
          <version>1.7.0</version>
      </dependency>

      <dependency>
          <groupId>org.scalatra.scalate</groupId>
          <artifactId>scalate-project_2.10</artifactId>
          <version>1.7.0</version>
      </dependency>



      <dependency>
          <groupId>org.eclipse.jetty</groupId>
          <artifactId>jetty-webapp</artifactId>
          <version>8.1.15.v20140411</version>
      </dependency>

      <dependency>
          <groupId>com.typesafe.scala-logging</groupId>
          <artifactId>scala-logging-slf4j_2.10</artifactId>
          <version>2.1.2</version>
      </dependency>

      <dependency>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-api</artifactId>
          <version>1.6.6</version>
      </dependency>

      <dependency>
          <groupId>ch.qos.logback</groupId>
          <artifactId>logback-classic</artifactId>
          <version>1.0.9</version>
      </dependency>


      <dependency>
          <groupId>net.databinder.dispatch</groupId>
          <artifactId>dispatch-core_2.11</artifactId>
          <version>0.11.1</version>
      </dependency>

Bug in AWS Image of Red Hat Linux 6.4

March 27, 2014

I was stuck a couple of hours with the following problem when using an Amazon Red Hat Enterprise Linux 6.4 (RHEL 6.4):

Configure and start the RHEL 6.4 Instance

Stop the instance, I was unable to connect again to that machine.

The problem is that the /etc/rc.local file inserts some configuration lines to /etc/ssh/sshd_config file and it has an error, so next time when machine restarts it fails to start up the ssh daemon.

The solution is to fix or remove the ssh lines from /etc/rc.local and fix the /etc/ssh/sshd_config file

Thanks to my colleagues which solved this issue.  

Whatsapp – Very impressive backend performance

February 20, 2014

Whatsapp Performance:

  • 571000 pkts/sec, >200k dist msgs/sec
  • Whatsapp servers are doing 1 million tcp connections on a single box: http://blog.whatsapp.com/?p=170
  • $ netstat -an | grep -c EST
    1016313

Whatsapp backend uses:

  • Dual Westmere Hex-core (24 logical CPUs)
  • 100GB RAM, SSD
  • Dual NIC (user-facing, back-end/distribution)
  • FreeBSD 8.3
  • OTP R14B03 (Erlang Open Telephone Platform)

http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf

 

https://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2011/

 

PS:

Akka and reactive programming is the way to go in server side!

Http Server Mock

February 13, 2014

Today I found nice open source framework mocking an Http Server

http://wiremock.org/

Really a nice one.

But I fought with the following exception when using it from Scala class:

java.lang.Exception: The @Rule 'wireMockRule' must be public.

And finally found a solution here:

http://randomallsorts.blogspot.co.il/2012/11/junit-411-whats-new-rules.html

Scala 99 Problems

January 16, 2014

Yesterday I started solving some interesting Scala riddles from this site:

http://aperiodic.net/phil/scala/s-99/

Here are some of my solutions

package MavenTest

import org.junit._
import Assert._

/**
 * Created by roma on 1/15/14.
 */
@Test
class Test1 {

  @Test
  def testNth1()
  {
    assert(2 == MyList.nth(2, List(1, 1, 2, 3, 5, 8)))
  }

  @Test
  def testNth2()
  {
    assert(2 == MyList.nth1(2, List(1, 1, 2, 3, 5, 8)))
  }

  @Test
  def testNth3()
  {
    assert(2 == MyList.nthRecursive(2, List(1, 1, 2, 3, 5, 8)))
  }

  @Test
  def testLength1()
  {
    assert(6 == MyList.length(List(1, 1, 2, 3, 5, 8)))
  }

  @Test
  def testReverse()
  {
    val res = MyList.reverse(List(1, 1, 2, 3, 5, 8))
    assertTrue(res sameElements List(8, 5, 3, 2, 1, 1))
  }

  @Test
  def testPalindrome()
  {
    val res = MyList.isPalindrome(List(1, 2, 3, 2, 1))
    assertTrue(res)
  }

  @Test
  def testPalindromeFalse()
  {
    val res = MyList.isPalindrome(List(1, 2, 3, 2, 1, 2))
    assertFalse(res)
  }
  @Test
  def testFlatten()
  {
    val res = MyList.flatten(List(List(1, 1), 2, List(3, List(5, 8))))
    assertTrue( res == List(1, 1, 2, 3, 5, 8))
  }

  @Test
  def testCompress()
  {
    val res = MyList.compress(List('a, 'a, 'a, 'a, 'b, 'c, 'c, 'a, 'a, 'd, 'e, 'e, 'e, 'e))
    assertEquals(res, List('a, 'b, 'c, 'a, 'd, 'e))
  }

  @Test
  def testFoldLeft()
  {
     val list = List(1,2,3,4,5,6,7,8,9,10)
     val res = list.foldLeft("")( (r, e) => r + "," + String.valueOf(e) ).drop(1)
     assertEquals("1,2,3,4,5,6,7,8,9,10", res)
  }

  @Test
  def testFoldRight()
  {
    val list = List(1,2,3,4,5,6,7,8,9,10)
    val res = list.foldRight("")( (e, r) => r + "," + String.valueOf(e) ).drop(1)
    assertEquals("10,9,8,7,6,5,4,3,2,1", res)
  }

}

object MyList
{

  def compress[T](list : List[T]) : List[T]=
  {
    list match {
      case Nil => Nil
      case h :: Nil => h :: Nil
      case e1 :: t if (e1 == t.head) => compress(t)
      case e1 :: t if (e1 != t.head) => e1 :: compress(t)
    }
  }

  def flatten[T, H](list : List[H]) : List[T] =
  {
     list match
     {
       case Nil => Nil
       case (h : List[H]) :: t => flatten(h) ::: flatten(t)
       case (h : T) :: t => h :: flatten(t)
     }
  }

  def isPalindrome[T](list : List[T]) : Boolean =
  {
     list match
     {
       case Nil => true
       case h :: Nil => true
       case h :: t => h == t.last && isPalindrome(t.init)
     }
  }

  def reverse[T](list : List[T]) : List[T] =
  {
      list.foldLeft(List[T]())( (rev, cur) => List[T](cur):::rev )
  }

  def length[T](list : List[T]) : Int =
  {
     list match {
       case Nil => 0
       case h :: Nil => 1
       case h :: tail => 1 + length(tail)
       case _ => throw new NoSuchElementException
     }
  }

  def nth[T](n : Int, list : List[T]) : T=
  {
     if (n < 0) throw new IllegalArgumentException("N")     var i=n     var e : T = null.asInstanceOf[T]     val it = list.iterator     while( i >= 0 && it.hasNext)
     {
       i=i-1
       e = it.next()
     }
    e
  }

  def nth1[T](n : Int, list : List[T]) : T=
  {
     if (n==0)
     {
        list.head
     }
     else
     {
        list match
        {
          case h :: x :: t =>
            System.out.println("h = " + h)
            System.out.println("x = " + x)
            nth1(n-1, x :: t)
          case _ => throw new NoSuchElementException
        }
     }
  }

  // Not that much harder without.
  def nthRecursive[A](n: Int, ls: List[A]): A = (n, ls) match {
    case (0, h :: _   ) => h
    case (n, _ :: tail) => nthRecursive(n - 1, tail)
    case (_, Nil      ) => throw new NoSuchElementException
  }
}