Search results for: data

Moving Past Infrastructure Limitations

// 04.20.2016 // Infrastructure

Here at MediaMath, we’re quite fond of data. It would be surprising to hear someone say they’re not fond of data, of course, but we’ve spent the last 18 months proving to ourselves and our clients that we really mean it. Our company is built around driving concrete, measurable results, and our clients – both internal and external – have sophisticated analytics teams that want access to the data we generate for their own analysis, owned marketing, budgeting, and more. In this post we will describe the journey from data warehouse to data platform and the success of ditching our […]

Scaling Data Ingestion Systems: From Perl to Go Part 1

A consequence of MediaMath’s astronomical growth over the past few years is dealing with huge growth in service usage. Rapid growth sometimes means that systems are built quickly, without making hard plans for the future. Systems with headroom can now often become insufficient in as little as six months, and so technical debt becomes a tough challenge to address. We deal with the question, “Do we try to re-write this, or do we modify what we already have to scale with the load we expect to see?” Nowhere has this been clearer than in ingesting user data, which since 2011 […]

Experiments with C and Go: Spec-ing out a new changelog

// 05.07.2015 // Platform API

Here at MediaMath, we store and act on a lot of data – many terabytes a day. A small subset of that data – all of the marketing campaign, ad unit, client, spend, billing, and reporting data, which is used for our digital marketing platform, TerminalOne (T1) – is stored in a PostgreSQL database. And while a changelog exists for this database, it was built early in MediaMath’s development and is tightly coupled with – and therefore reflective of – only select core T1 API endpoints, ignorant to other endpoints entirely. We needed to build a new service that could […]

Data Liberation at MediaMath

// 04.15.2015 // Data

MediaMath was recently at Amazon Web Services Re:invent 2014, where we presented on our open data platform and data liberation project, both of which are enabled by a variety of tools including many AWS tools. Below is a recording of our presentation: Data Liberation at MediaMath. Aggregating and processing terabytes of data per day is a challenge for any technology company. As marketers and brands become more sophisticated consumers of data, enabling granular levels of access to targeted subsets of data from outside your firewalls presents new challenges. In this presentation, VP of Engineering Edward Fagin and Senior Director of Data […]

Extending Play’s validation to work with Big Data tools like DynamoDB, S3, and Spark

// 03.18.2015 // Data

In this two-part blog series, we are looking at how MediaMath uses Play’s API to perform data validation on big data pipelines. In part one, we covered data validation with Play’s combinator-based API. In part two, we’ll extend that data validation to work with Amazon Web Services DynamoDB, AWS S3, and Spark. Extending validation to work with AWS DynamoDB MediaMath uses a variety of technologies in our analytics stack, including AWS DynamoDB. DynamoDB is a distributed, fault-tolerant key value store as a service that makes it easy to store/query massive datasets. We use it to power a few internal troubleshooting […]

Scaling data tools: How Play enables strongly typed big data pipelines

// 03.04.2015 // Data

The other day, I was talking with a colleague about data validation, and the Play web framework came up. Play has a nice API for validating HTML form and JSON submissions. This works great when you’re processing small amounts of data from the web-tier of your application. But could that same tech benefit a Big Data team working on a backend powered by Hadoop or Spark? We decided to find out, and the results were encouraging. The secret sauce? Play’s combinator-based approach to data validation. Whether your data is big or small, garbage in is garbage out MediaMath processes TBs […]

Page 1 of 3123