Skip to content
Contact Us
  • There are no suggestions because the search field is empty.

Internal Monitoring

Internal monitoring tracks how each of your AI use cases performs after it is deployed. You choose what to measure, collect values over time, and see when evidence is missing or a threshold is crossed. This gives you a record that a use case is still operating the way it was approved. 

How Internal Monitoring works

Internal monitoring is different from external monitoring. External monitoring watches for outside events, such as incidents, vendor changes, and new regulations. Internal monitoring looks at the measures from your own deployed use cases. It is also not a security tool and does not block or respond to activity in real time.

Stakeholder Function

  • Governance leads use it to confirm a use case is still working as intended after approval, instead of tracking it in spreadsheets and chasing owners by email.

  • Audit and quality teams use it to keep documented performance measures, evidence, owners, and a collection schedule for each use case.

  • Technical teams use it to keep the observability tools they already have and connect those results to the governed use case.

  • Risk owners use it to learn when a metric crosses a threshold that matters for their use case, so they can investigate or start a review.

Define measures for each use case

Choose the metrics that matter for the use case. Measures can cover business outcomes, agent behavior, quality, compliance, and ethical AI. Common examples include cost, usage, drift, reliability, bias, error rate, and completion rate. You can also create custom metrics.

Simple measures are valid. A monthly invoice total or a count of user complaints is a good place to start.

Set a schedule, an owner, and thresholds

For each metric, set how often it should be collected and who is responsible. You can also set a threshold, so you are alerted when a value goes above or below the level you set. Trustible tracks evidence submissions and shows which ones are overdue.

Collect values in the way that fits

You can bring in metric values three ways:

  • Review workflows. A person answers a recurring task with the latest value.
  • API integrations. Results from your existing monitoring tools are sent to Trustible, through an API or webhook.
  • Manual input. A business owner enters or uploads a value. They get an email when an upload is due.

Some measures need a person to judge them, such as whether a response was correct. Others, like cost per request, may need an invoice upload or an API connection.

See history and evidence

Collected values build up into a time series, so you can see how a metric changes. Because Trustible keeps the values you collect, you can look back further than many source tools allow. You can also attach supporting evidence to explain a change in a metric.

Things to know before you set it up

  • Start small. Many teams begin with their higher-risk use cases and a few simple measures, then add more over time.
  • Periodic review is a separate feature. You can set up a review task for intended-use changes and quality updates, and a dashboard shows upcoming review dates. Internal monitoring is available on use case records, but it is not currently a workflow task. Treat monitoring and periodic review as two connected features, not one automatic flow.
  • Threshold alerts and reviews. Trustible can alert you when a threshold is crossed. Automatically starting a review from an alert is not something to count on yet.

Frequently asked questions

Is internal monitoring the same as the AI incident feed?

No. The incident feed matches outside incidents to your use cases. Internal monitoring collects measures from your own deployed use cases. Push notifications for incidents are not live yet.

Do I have to automate every metric?

No. You can mix manual entry, review-based collection, and API collection. Use whichever fits each metric.

Does Trustible replace our security or observability tools?

No. Trustible keeps the governance record: documentation for audits and performance over time. Security tools handle enforcement, guardrails, and real-time response. Your existing monitoring tools can send their results into Trustible.

Does Trustible store raw logs?

Not in the first version. The approach is to pull a metric or snapshot from the source system and store that value. Check with the Trustible team for the current setup before making commitments about architecture.

Will a threshold breach automatically start a review?

You will get an alert. Automatically starting a review is a goal for the future. Check with support for what is available in your account today.

Is internal monitoring included in my plan?

Confirm what is in your package with your Trustible contact. For questions about setup or availability, email support@trustible.ai.