Fairo takes the complexities of your AI systems and strategy and breaks them into manageable resources. These resources can be easily accessed and modified by both our REST API and our web application. With Fairo you have complete flexibility in how you use our system and integrate it into your ecosystem. Our goal in designing these docs is to be as transparent and clear as possible

Fairo APIs

There are 4 main APIs offered by Fairo. Each API contains a set of related features designed to help your organization and your users achieve a specific goal of set of goals. Our APIs all following the same basic structure and design pattern. For information on specific API endpoints and API requests please consult our API Reference.

The Core API

Fairo's Core resources are the building blocks of the AI governance process. With them, you can document how AI is developed, used, tested, and deployed within your organization. Each resource is designed to serve a specific purpose in the process of building an auditable AI system.

In the following sections, we will go over the core resources and describe what purpose they serve and how you can best leverage them to meet your needs.

Roles

A role in Fairo is used to associate a list of users with a certain 'real world' role. Some example roles that you could use in Fairo could include:

Software Engineer
Data Scientist
Data Analyst
Chief Scientist
Release Engineer
Cloud Engineer
Product Manager
Etc...

Roles are used throughout the system to help us contextualize how individuals are contributing to the AI development lifecycle and also to the AI governance process as a whole. Understanding what role an individual plays in each step of the process helps the organization set clear responsibilities for individuals. Understanding roles and responsibilities in any process, be it product development, software testing and release, or AI governance is essential.

Roles can be associated with very specific sets of responsibilities that are unique to your company. You should feel free to get creative with the role resource, and leverage in a way that helps individuals within your organization operate more decisively.

Actions

The actions resource describes actions that individuals who assume a certain role perform. Actions can have one or more roles associated with them. Actions on their own are just descriptive. Meaning, when you create an action resource you are describing something that someone assuming a certain role does, but you are not saying that they did that thing. To learn more about how Fairo handles instances of action, see the Task section under Process Execution.

Actions have 4 main categories:

Review
A user is asked to review something, a user can leave comments and mark as complete.
Approval
A user is asked to approve something, a user can leave comments and mark as approved.
Jira
This action is associated with work that occurs outside of Fairo on the Jira platform. When Fairo tasks are created they can be linked to Jira issues from within Fairo (using UI or API) or from within Jira, by adding "Fairo:Task-" to the issue. You can also link manually in the task view or with the API.
GitHub
This action is associated with work that occurs outside of Fairo on the GitHub platform. When Fairo tasks are created they can be linked to GitHub issues from within Fairo (using UI or API) or from within GitHub, by adding "Fairo:Task-" to the PR or issue description, or to a comment on the PR or issue. You can also link manually in the task view or with the API.

Other External Action
This action can be associated with any plugin system where actions are documented within your organization. This could be a process automation tool, a task management tool, a compliance tool, or a basic word or excel document. Fairo's plugin system is designed to allow you to document actions as flexibly as required to meet your organization's requirements.

Using the action resources, you can to contextualize the work that is occurring within your organization, regardless of where it occurs.

Assets

Assets in Fairo Core are highly flexible containers of information that can be applied to numerous use cases. Each asset has a type. The default set of asset types is:

Documentation
Policy
Dataset
Algorithm

However, using the asset_type resource, you can create custom asset types that fit the needs of your organization.

With this implementation, users can to document which assets they are managing as part of their AI development lifecycle. If users want to add more context/details to their assets, they can upload files, create custom fields, or link plugin data to the asset.

Custom Fields

The custom field resource the combination of a field name and data type. The supported data types, with some ideas for how to use them, are:

Field Type	Description	Examples
Date	A valid date value.	internal due date, external due date, expiration date, etc.
String	A valid string value.	external object ID, alternate resource description, geo-tag, etc.
Number	A valid numeric value.	required compliance percentage, expected revenue, required reviewers, etc.
JSON	A valid JSON value.	AI training output, custom metadata, contexts, etc.
User	A valid user ID.	internal auditor, external auditor, etc.
Fairo Asset	A valid Fairo asset ID.	i.e. policy, documentation, models, etc.
MLFlow Object	A reference to an MLFlow asset ID.	MLFlow models, datasets, assets, etc.

Once you create a custom field resource, then values can be associated with it. If you do not create a custom field, you cannot create custom field values.

Custom Field Values

Custom field values assign values to custom field resources, and also to any other resource in Fairo. For example, if I create an asset resource, I can associate a custom field value with it by adding the ID of that resource to the custom field value when creating the custom field. With this pattern, custom fields are extremely flexible.

Files

The file resource is a simple file upload/download tool that allows you to associate files with resources in Fairo. Fairo calculates the amount of storage you are currently using. To see the amount of storage you are using use the total_storage_gbs resource. There is technically no limit to the amount of storage you can use, however, we do plan to charge per GB/Month and will be forced to start charging sooner if excessive storage is used.

Workflows

The workflow resource is the entry point of the Fairo Core API. All data related to how your company is building and governing AI is contained under workflows. Workflows are designed to make it easy for you to categorize the work your organization is undertaking as part of their AI governance process. Because of the inherit complexity of the AI development lifecycle, having the workflows resource will help you stay organized when using Fairo.

Process Graph

The process graph is the central Fairo resource. Process graphs are contained under workflow and can be thought of as real-life flow charts. When you build a flow chart describing a process or a set of procedures, it is generally a static image. The Fairo process graph is anything but static.

Process Node / Process Edge

The process node and process edge are what construct the Process Graph in Fairo. By creating a creating a set of nodes/edges, you are constructing a graph. Within the process node, you can link to other resources within Fairo to help bring your process graph to life. This can be done via element mappings, or by using the dedicated asset, action, and test case mapping feature (see API reference or Fairo application).

Through these mappings, we can begin to visualize the Fairo task graph as a real-life pipeline comprised of people data, tests, actions, and results. This goes beyond a traditional data pipeline that is generally thought of as just processing data. Fairo workflows can reach across your entire organization if configured to do so.

When building process graphs in Fairo there are very few limitations. For each Fairo resource that is associated with a process node and processes graph, the audibility of your AI system is enhanced.

Process Execution Plan

After creating detailed roles, assets, test cases, actions, and process graphs, it's time to do some monitoring of real work. Up until this point, everything has been more or less abstract or descriptive. Meaning, we're not saying someone 'did' an action, performed a test, or adhered to a process.. all we are saying is that if they were to do so, this is what it would look like.

The process execution plan resource is where the rubber meets the road for all Fairo resources. With this resource, you can set dates around when a process graph is being executed on, and document who is responsible for completing the work associated with each process node through process execution items.

Process Execution Item

The process execution item is an executed process node. When using a process execution item, you can assign a , set start date, end date, due date, and completed date, link multiple users as collaborators. Update default custom fields from the process node with execution specific data, and create tasks associated with any actions that are related to the process node.

Tasks (Process Execution Tasks).

The task resource is designed to assign a user to work associated with an action. Because tasks are associated with specific actions in Fairo, they inherit their type.

Tasks behave slightly differently depending on the type of action to which they are associated. For reviews and approvals, a task only needs some notes and comments to be marked as complete or approved. For tasks that are linked to Jira and GitHub, users can add issues to the task resource from within Fairo or from the external system.

To add data from Fairo, simply add the object ID from your Jira project or Github repo to the API or UI. To add from Github or Jira, navigate to the issue and add in the summary, description, or comments (GitHub only), Fairo:Task-.

The Test Suite API

The Test Suite API will allow you to test your AI with ease. In addition to providing a full test management system, we also provide the concept of metrics. In the following sections, we will go over the different resources offered in the Test Suite API.

Metrics

Metrics are an important resources in the Fairo platform. The metrics resource is designed to store key information about the metrics your organization uses to evaluate its AI systems, datasets, and more. In addition to allowing users to create their own metrics, and add metric references (citations of sources for the metric), Fairo also provides pre-built and referenced metrics on our platform, (Python API)[https://fairo.readme.io/docs/introduction], and REST API.

Parameters

Parameters are a useful feature for metrics. Parameters allow you to understand what data was sent to metrics to give a particular result. For AI testing, this frequently includes which models and which data tests the metric was computed on.

Metric parameters can be optional or required. All required parameters are required to be passed when creating metric results.

Test Cases

The test case resource in the Fairo Core API is the foundational resource of the Fairo Test Suite. With the test case resource, you can document the test cases that you use in your AI testing process. The test case, like all other Fairo resources, is linkable to our workflow and process graph resources.

Parameters

Parameters are a useful feature for test cases. Parameters allow you to understand what data was sent to test cases to give a particular result. For AI testing, this frequently includes which models and which data tests were applied to.

Test parameters can be optional or required. All required parameters are required to be passed when creating test results.

Note: valid parameters value types include all custom field value types.

Automatic Testing

The test case resource has some very interesting features designed specifically for AI testing. You can set an automatic test operator and test value. The valid test operators are.

Operator	Name	Supported Value Types
>	Greater than	Numeric, Date
<	Less than	Numeric, Date
>=	Greater than and equal to	Numeric, Date
<=	Less than and equal to	Numeric, Date
==	Equals	Numeric, Date, String, JSON, Asset
!=	Not equal	Numeric, Date, String, JSON, Asset
startswith	Starts with	String
sndswith	Ends with	String
contains	Contains	String

When you set an automatic test operator and test value, test result status will compute automatically. If you don't want to automatically test, do not set a test value or operator.

Associating Metric Results

Test results can be further contextualized by association of metric results to test results. When a test case has an associated metric, users can associated computed metric results with the test result. For example, if you wish to test that ROC > .8, you can create a test case associated with ROC metric, set test operator as > and test value as .8 and you are ready to test. All of this can be configured via the API and easily embedded into existing processes.

Test Run

A test run is a collection of test cases and test results. Test runs are designed to group test results and test cases for specific goals or business objectives. For example, if you have to run a pre-release set of fairness tests, that is what a test run would be created for.

Test runs are also useful for grouping and visualizing test results in relevant groupings.

The MLFlow API

The MLFlow API is a fully managed version of MLFlow, an MLOps tool for managing the machine learning lifecycle. For details on the MLFLow API, Python packages, etc. please consult the official documentation official documentation

The Compliance API

The compliance API is designed to help your organization contextualize data from the Fairo system in order to understand your risk, governance, and compliance posture. Resources in this API will help you document controls, risks, use-cases, evidence, and more. Many of the resources available in this API are provided already by Fairo, for example, the NIST Risk Management Framework.

Mapping evidence and elements across your system to the compliance resources can be easily automated by the API. Other use cases include importing or exporting data from the system related to controls, risks, evidence, etc.

Registries

Registries are flexible access-controlled repositories where your organization can store AI governance data. The registry tagged with a registry type, and a set of registry fields. Registry fields extend the following base types:

Field Type	Description
Fairo or MLFlow Object	Any object (from any API resource) in the Fairo or MLFlow system.
Fairo or MLFlow Object List	A list containing many objects of the same type in the Fairo or MLFlow system.
Number	A numeric value.
Number List	A list of numeric values.
Date	A valid date value.
Date List	A list of date values.
String	A string value.
String List	A list of string values.
JSON	A valid JSON value.
JSON List	A list of valid JSON values.
Fixed Choices (i.e. High, Low, Medium)	A set of fixed choices (levels) pre-programmed by the user.

Then as items are added, the fields are populated. Each registry item is tagged with a version and has an audit and revision history. This allows for a clear, immutable, record of the evolution of the registry over time.

Compliance Elements

Compliance elements are the building blocks of the Fairo compliance system. This flexible resource allows you to store all relevant controls, concepts, principles, and additional elements that relate to governance risk and compliance in your organization.

All compliance elements live in a global namespace called the library. This allows elements to be mapped to any resource, framework, or compliance element in the Fairo system. This gives you a full view of your risk, governance, and compliance posture across all AI elements in your organization.

Element Mappings

Element mappings are used to create relationships across all resources in the Fairo system. Mappings are used extensively when building compliance frameworks, such as the NIST Risk Management framework. Element mappings are also used for linking evidence to evidence requirements and are basis of automatic-evidence gathering.

Evidence Requirements

Evidence requirements are generally associated with controls. Every evidence requirement has an associated set of tests/criteria that must be satisfied within a certain time constraint in order for the requirement to be marked as completed.

Some evidence requirements are associated with actions, such as reviews, approvals, or externally documented actions. In these cases, Fairo will create tasks (either inside Fairo, or using a plugin of your choice) that will ensure these actions are completed in a timely manner. Fairo will also highlight any overdue actions or evidence requirements that are not satisfied within the required time frame.

Compliance Frameworks

Compliance frameworks are collections of compliance elements. An example of a common framework for AI governance is the NIST AI Risk Management Framework. The Fairo framework view looks across all frameworks in the library to check which evidence requirements and controls are fully satisfied, which are partially satisfied, and which are not completed and overdue.

This resource and corresponding views can be configured for external Auditors looking to leverage the system, or exported into a compliance tool of your choice via the API.