Search results for: Developer Best Practices

How We Used Docker to Lower Test Run Times from 1 Hour to 10 Minutes

When a service grows in size and complexity, we add more tests in order to maintain test coverage. Having proper test coverage allows us to change or add new features and be reasonably confident we didn’t break any existing features. This is especially important for “bidder”, the name of our real time bidding service, where even a small unexpected downtime or bug can have major consequences. Bidder interacts with ad exchanges through http requests to place bids on advertisement opportunities (webpages, mobile apps, etc.) for our advertisers. As bidder increased in features and handled more bid opportunities (millions of bid […]

When You’re Done, You’re Only Halfway Done

You’ve just checked in your code, given it a version number and are about to send it to QA, but wait, are you sure you are done? Sometimes when I use a new whizbang program, only to find it lacks some things which every useful program should have.  It is easy, especially when building a complex program with unique functionality, to focus entirely on the unique offering of the program. While an enormous amount of time should be put into differentiating a program, there are certain features that users now expect in order to get the highest performance out of […]

Counting at Scale: HyperLogLog to the Rescue

MediaMath processes many terabytes of data each day for the various reports available in T1. One metric we show is the number of unique impressions for each campaign, there is a big difference between showing an ad to 100 different people and showing the same ad to one person 100 times. While this is conceptually a simple problem, solving it at scale is not quite as straightforward. The canonical way of solving this problem would be for any given campaign to put the id of each person who saw an ad for that campaign into a set and then check […]

Functional Error Accumulation in Scala

This post originally appeared on longcao.org in July 2015 and is republished below. In a previous post, we dipped into what it means to handle application errors functionally in Scala and took a brief overview of the types that can be used to accomplish that. A common theme with those error handling types like Try  or the disjunctions is that they all fail fast: only the first error in a purely functional call chain is returned. Let’s keep the functional hype train going and explore more of the goodies that we can bring into our application logic. Stepping back from the […]

Easing into Functional Error Handling in Scala

This post was originally published on longcao.org in June of 2015 and is republished below. One of the biggest selling points of Scala is being able to transition from Java to the world of functional programming. What this means, practically speaking, is that you and/or your team can start off writing very Java-like Scala that already works, treating it as a “better Java” with more goodies and less verbose syntax. But why stop at that? We can do better by adopting more typesafe functional concepts, and the best part is we can ease ourselves into at least one practical application […]

Why You Should Not Rely on StatsD for Monitoring or Optimizing Response Time

In my blog post last week, I outlined the difficulties and importance of accurately measuring and reporting response times, as they are critical for ensuring customer happiness. This week, I will be taking a look (and a rant) at StatsD, a commonly used tool for systems monitoring. As a brief refresher, measuring and collecting response times can result in a huge number of metrics. A system that sees 70k RPS (requests per second) generates over 4 million response time measurements every hour. Statistical analysis is required to visualize or summarize these measurements, which inherently introduces a measure of subjectivity. Depending […]

Page 1 of 212