Metrics

Tracking metrics allows for deep understanding in how the system is operating and creates baseline understanding on what is normal. Consistent tracking of metrics can help in early anomaly detection.

The Integrity application, if configured to do so, generates a number of metrics. Each metric has a name and a type, e.g. Counter, Gauge, or Timer. The name and type determines a number of different time series exposed by a particular metric. These time series can be monitored and analyzed, using supported monitoring backends. Currently, Integrity supports JMX and Prometheus.

The type, naming conventions and number of time series generated from a particular metric may vary depending on the selected monitoring backend.

Enabling metrics

Metrics are contfigured on the server object configuration in "config.json".

"server":

In options object metrics are enbled using JMX and/or Prometheus.

"options":{ "metricsOptions": {
    "enabled": true
}

Monitoring systems

Additionally one or more metrics system must be enabled.

JMX

 "options": {
        "metricsOptions": {
            "enabled": true,
            "jmxMetricsOptions": {
                "enabled": true
            }
        }
    }

Default JXM setup does not allow for remote jxm connection. To enabled startup options must be set:

Add the following JVM arguments when starting your Vert.x application to enable remote JMX access:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=PORT_NUMBER 
-Dcom.sun.management.jmxremote.rmi.port=PORT_NUMBER 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.local.only=false 
-Djava.rmi.server.hostname=SERVER_IP_ADDRESS

PORT_NUMBER is the port you want to use for JMX (e.g., 9010).

SERVER_IP_ADDRESS should be set to the IP address where the server is reachable.

Note: If you enable remote access in a production environment, you should set authenticate=true and configure a password file for security, as well as apply firewall rules to restrict access.

This configuration enables JMX metrics in Vert.x and allows remote access through the JMX protocol.

Ensure that the appropriate firewall ports are open and that the connection is properly secured.

Prometheus

Basic setup

{
        "metricsOptions": {
            "enabled": true,
            "prometheusOptions": {
                "enabled": true,
                "startEmbeddedServer": true,
                "embeddedServerOptions": {
                    "host": "0.0.0.0",
                    "port": 9090,
                    "embeddedServerEndpoint": "/metrics"
                }
            }
        }
    }

Using SSL

{
    "metricsOptions": {
        "enabled": true,
        "prometheusOptions": {
            "enabled": true,
            "startEmbeddedServer": true,
            "embeddedServerOptions": {
                "host": "0.0.0.0",
                "port": 9090,
                "ssl": true,
                "keyCertOptions": {
                    "pemKeyCertOptions": {
                        "keyPath": "config/server-key.pem",
                        "certPath": "config/server-cert.pem"
                    }
                }
            },
            "embeddedServerEndpoint": "/metrics"
        }
    }
}

Available Metrics

Here is a list of fixed metrics exposed by the Integrity application.

Name	Labels	Type	Description
ffid_pipes_pending	id	Gauge	Number of outstanding requests
ffid_pipes_processed	id	Counter	Number of requests returned successfully
ffid_pipes_failed	id	Counter	Number of requests returned failure
ffid_pipes_errors	id	Counter	Number of requests returned error
ffid_pipes_processed_duration	id	Timer	Success processing time in milliseconds
ffid_pipes_failed_duration	id	Timer	Failure processing time in milliseconds
ffid_pipes_errors_duration	id	Timer	Error processing time in milliseconds
ffid_valves_processed	id, class	Counter	Number of requests returned sucessfully
ffid_valves_failed	id, class	Counter	Number of requests returned failure
ffid_valves_processed_duration	id, class	Timer	Success processing time in milliseconds
ffid_valves_failed_duration	id, class	Timer	Failure processing time in milliseconds
fortifiedInternalSessionCount	id	Gauge	Number of active sessions.

Dynamic metrics

Apart from fixed metric names Integrity exposes a number of dynamic metric names. What tis means is that metrics will be exposed based on naming and configuration of the current system setup. The idea behind this is to allow for detailed monitoring if desired.

Dynamic metrics, in general, comes from modules that have outbound relations (http, ldap sql ,smtp etc), Pipes module (valves) & authenticators. Details in regards to metris is found in respective section av the documentation.

Vert.x

The Integrity application is largely built on Eclipse Vert.x. Vert.x generates metrics related to the Vert.x event bus, monitoring low-level activity on the application level.

Metrics generated by Vert.x can also help monitor and analyse incoming/outgoing traffic over various protocols, for example:

• HTTP activity

• TCP activity

• Datagram transmission and reception

Metrics exposed by Eclipse Vert.x are documented here:

https://vertx.io/docs/vertx-micrometer-metrics/java/#_vert_x_core_tools_metrics