[CDAP-12070] Finer control over whether to emit metrics Created: 07/Jul/17  Updated: 22/Aug/17  Resolved: 22/Aug/17

Status: Resolved
Project: CDAP
Component/s: App Fabric
Affects Version/s: 4.2.0
Fix Version/s: 4.3.0

Type: Improvement Priority: Major
Reporter: Andreas Neumann Assignee: Vinisha Shah
Resolution: Fixed Votes: 0
Labels: blue, operability

Issue Links:
relates to CDAP-12427 Document Metrics enable/disable option Resolved
Release Notes: Added option to disable/enable metrics for a program through runtime arguments or preferences. This feature can also be used system wide by enabling/disabling metrics in cdap-site.xml
Rank: 1|i004qf:


Metrics can cause a high load on HBase (especially in an already overloaded cluster). When this happens, it would be good to have the option of disbling metrics until the system quiets down.

Since the load on the metrics system is proportional to the number of running containers, high load is really only generated by application containers (for example, a MR with 1000s of tasks).

The CDAP system only runs a handful of edge nodes and less than a dozen system services. It is desirable to keep system metrics always on. But it is also desirable to disable metrics from applications, especially programs that have many tasks. Therefore we will add an option

  • in cdap-site.xml (system wide) to control whether app containers emit metrics
  • as a program argument to override the system-wide setting per program run, or permanently for a program by setting a preference.

This will allow disabling/enabling program metrics through a preference without requiring a Master restart.

Comment by Shankar Selvam [ 16/Aug/17 ]

System level change to enable/disable metrics collection was done in : https://github.com/caskdata/cdap/pull/9358

Comment by Vinisha Shah [ 21/Aug/17 ]

PR: https://github.com/caskdata/cdap/pull/9434

Generated at Wed Feb 20 13:16:23 UTC 2019 using Jira 7.13.0#713000-sha1:fbf406879436de2f3fb1cfa09c7fa556fb79615a.