Currently metrics from mapreduce tasks are reported using mapreduce framework counters. This looks like a hack:
- counters info has to be reported as gauges by mr submitter
- there are limitation on what you can do with counters (incl counter name size, and number of counters, e.g. see
- some of metrics data transfer is redundant
Instead, what we want is tasks to report metrics as any other YARN container that executes any other program: directly to Kafka.