Add Application Insights telemetry for CLI observability

[petehauge]

Summary

Integrate Azure Application Insights into apiops-cli to enable observability, diagnostics, and usage analytics. This will help the team understand how the CLI is performing in aggregate and identify emerging issues proactively.

Goals

• Crash/Error Reporting: Automatically capture unhandled exceptions, stack traces, and error conditions
• Performance Monitoring: Track command execution times, identify slow operations or performance regressions
• Hang Detection: Monitor for commands that take unexpectedly long or become unresponsive
• Usage Analytics: Understand which features are being used and adoption patterns
• Version Tracking: See which versions of apiops-cli are in active use

Telemetry Requirements

Command Usage Metrics

Track custom events for each command execution:

• apiops extract - count per month
• apiops publish - count per month
• apiops init - count per month (understand adoption of new commands)
• Include CLI version as a property on all events

Error/Exception Tracking

• Capture unhandled exceptions with full stack traces
• Track handled errors with context (command, arguments, environment)
• Include correlation IDs for debugging

Performance Metrics

• Command start/end timestamps
• Total execution duration per command
• API call latencies (optional, for deeper analysis)

Technical Details

SDK

• NPM package: applicationinsights
• Docs: https://github.com/microsoft/ApplicationInsights-JS

Example Usage (from Alexander)

var snippet = {
    config: {
        connectionString: "InstrumentationKey=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
};
var init = new Microsoft.ApplicationInsights.ApplicationInsights(snippet);
var appInsights = init.loadAppInsights();

// Track custom events
appInsights.trackEvent({
    name: 'command_executed',
    properties: {
        command: 'extract',
        version: '1.0.0',
        duration_ms: 1234
    }
});

Configuration

• Instrumentation key will be provided separately (not in source code - use environment variable or embedded config)
• AppInsights instance: "APIops" (currently in APIC subscription, may move to dedicated sub later - key won't change)

Privacy Considerations

• Do NOT collect sensitive data (API keys, connection strings, file contents)
• Consider opt-out mechanism for users who don't want telemetry
• Document what telemetry is collected

Implementation Notes

• Initialize App Insights early in CLI startup
• Wrap main command execution in try/catch to capture crashes
• Use trackEvent for command usage, trackException for errors
• Flush telemetry before process exit to ensure data is sent
• Consider batching/sampling for high-frequency operations

References

• Recommendation from: Alexander Zaslonov
• SDK: https://github.com/microsoft/ApplicationInsights-JS
• NPM: https://www.npmjs.com/package/applicationinsights

[EMaher]

Incorrectly associated with mcp support PR. Reactivating

[petehauge]

We don't have a way to add app insights without including the instrumentation key somewhere. So for now we're going to rely on the user-agent string for telemetry only.