Search results for: February 2015

Raise a glass to TOAST, a preprocessing service converting XML to JSON

TOAST is a new project recently released by the UI team here at MediaMath. The name stands for “TerminalOne App Server TOAST,” following the long-standing tradition of recursive acronyms in software (and adding to a relatively shorter list of more “exclusive” tail-recursive acronyms … but I digress). “What is TOAST really?” one might ask. The answer: TOAST is a web service layer that sits between the T1 front-end (the UI on our primary product, TerminalOne) and the services upon which the UI relies. It may ultimately provide a number of features for T1 that, typically for performance reasons, are preferably […]

Cut your run time from minutes to seconds with HBase and Algebird

// 02.04.2015 // Data

[Note: Code for this demo is available here: https://github.com/MediaMath/hbase-coprocessor-example] At MediaMath, our Hadoop data processing pipelines generate various semi-aggregated datasets based on the many terabytes of data our systems generate daily. Those datasets are then imported to a set of relational SQL databases, where internal and external clients query them in real time. When a query involves extra levels of aggregation on an existing dataset at run time, it starts to hog server resources, slowing down runtime. However, we have been able to reduce the query time on these terabyte–scale datasets from minutes to seconds by using a combination of […]