AI governance

Your AI governance, built into every app.

Every Jaali app is AI-enabled. How much AI is used, in what ways, and where humans are in the loop are all up to you. Meanwhile, the platform ensures that the critical functions of your AI governance are preserved: protecting privacy, security, and accessibility; honoring human judgment and fairness; and maintaining transparency and control.

What we mean by AI governance. AI governance is how an institution decides where and how AI is used, and how it makes sure those decisions hold. Jaali enforces the critical functions of that governance in every app, and supports the rest.
Overall structure

How we think about AI governance

Using AI policies from select universities; guidance and frameworks from organizations such as EDUCAUSE, the National Institute of Standards and Technology (NIST AI Risk Management Framework), and the International Organization for Standardization (ISO/IEC 42001); and learnings from our incubating organization, AQL Labs, we developed a structure for how we answer the question “What must be in place for an institution’s AI use to be well governed?”

Our response is 15 elements organized into four areas. This gives us an anchor for what we demand of our apps and those we employ in service to this work, our own responsible-AI policy, and our responses to relevant components of HECVAT. Moreover, perhaps most importantly, this gives us a robust starting point for ensuring alignment with an institution’s specific AI governance policy.

Who is responsible: The institution determines Jaali enforces

1.Direction

Who decides, and on what basis

Principles and policyClear AI principles and policy guide every use of AI and connect to the institution’s existing policies.
Accountability and oversightNamed people and committees own decisions about AI, with clear processes for approval and appeal.
Risk and use-case reviewEvery use of AI is assessed for risk before launch, reviewed by the institution, and reassessed as it changes.

2.Protection

What every AI system must safeguard

Data protection and privacyData is classified and protected in line with FERPA, it is never used to train models, and outputs are treated as being as sensitive as their inputs.
Security and resilienceAccess to AI tools is tied to verified identity, and the tools hold up against misuse and attack.
Accessibility and equitable accessEvery AI tool meets accessibility standards (WCAG 2.1 AA), and access to AI is fair across the campus.

3.Responsible use

How AI is used with people

Human judgmentPeople review, verify, and approve what AI produces, and consequential decisions stay with people.
Transparency and disclosureAI involvement is labeled and its sources are shown, so people know when and how AI is used.
Fairness and impactAI is tested for bias, and its effects on different groups are monitored and managed.
Integrity and intellectual propertyAcademic and research integrity, copyright, and authorship are protected when AI is used.

4.Assurance

How the institution knows it’s working

Vendors and value chainVendors and model providers are vetted, and contracts are clear about how they handle data and risk.
Life-cycle control and recordsAI tools are tested before release, changes are controlled and versioned, and an audit log records what happened.
Models, cost, and environmental impactThe models in use, what they cost, and their environmental footprint are visible, and limits are in place.
People and capabilityStaff, faculty, and students have the AI literacy and training to use AI well.
Incidents and improvementProblems are reported and resolved, the right people are notified, and governance is reviewed regularly.
Governance in practice

How structure maps to practical rules

We’ve mapped the areas of AI governance above to 10 conditions and guardrails across three groups that answer the question: What rules does every Jaali app adhere to that ensure it lives up to the right standards of AI governance?

Who is responsible: The institution determines Jaali enforces

Protecting privacy, security, and accessibility

  • Security: Each app can encrypt data with the institution’s own keys and sign people in through the campus’s single sign-on, both configured by the institution.
  • Data protection: Each app keeps institutional data out of model training, follows the institution’s retention settings, and treats outputs as being as sensitive as their inputs.
  • Accessibility: Each app meets WCAG 2.1 AA, and no change can make it less accessible than it was.

Honoring human judgment and fairness

  • Human judgment: Each app lets AI suggest, summarize, or draft, and leaves review, approval, and consequential decisions to people.
  • Fairness: Each app builds on people’s strengths, never on assumptions drawn from who they are, and is tested for bias before release.

Maintaining transparency and control

  • Transparency: Each app labels AI involvement, shows its sources, and reports where and how AI is used.
  • Audit logging: Each app keeps a permanent record of who accessed what, what AI was asked, and what changed, which no one can edit or switch off.
  • Version control: Each app lets the institution track changes, engage version control, and revert to former builds. Users log the versions they want kept, and each logged version stays available. Reverting restores an app’s earlier build, but data a change deleted or transformed may not come back.
  • Model and cost controls: Each app shows what its AI costs, uses only the models the institution allows, stays within the quotas it sets, and can have AI switched on or off.
  • Incident response: Each app lets AI features be switched off quickly if something goes wrong, and tells the institution what happened and what was done.
Our responsible-AI commitments

How Jaali holds itself accountable

Governing AI well is also our job as a company. We are still finalizing the policy and putting in place the infrastructure to deliver on all of these, but Jaali has committed to the following and will make as much public as possible about how it holds itself accountable:

A published responsible-AI policyHow we identify, measure, and manage AI risk, reviewed every year.
A risk assessment for every appMade before launch and kept current as the app changes.
Annual training for everyone at JaaliA responsible-AI curriculum covering privacy, accessibility, fairness, and AI risk.
Clear terms with model providersDocumentation on retention and no-training terms for every model we offer.
A written incident processHow we detect problems, switch AI off, tell you, and review what happened.
An annual reviewOf our policy, rules, risks, and training, with a record of what changed.

Note that we take our AI governance very seriously. We are still incorporating and testing the rules we lay out above to be sure that they are indeed as embedded as we have articulated.

Let's talk about the work.

Tell us what is taking your team’s time. That is where every Jaali conversation starts and we would be delighted to have that conversation with you.

info@jaali.ai