7 Minute Read

Credit Agreement Analysis with CreditSeer

Most designers stop at the prototype. I built a lightweight extraction pipeline first, to design around what AI could extract, miss and hallucinate in credit agreements.

Duration
6 months (Scope->MVP)
Team
1 PM, 2 ML, 1 SDE
Role
Product Design Lead
Problem

GT Financial Services Innovation Lab partners with Atlanta-area regional banks to help speed up their loan turnaround process.

Before a credit analyst can assess risk of a borrower, they spend ~2–4 hours manually navigating  100–200 page documents. GenAI can make this faster, but if it hallucinates a value, that’s a compliance risk.

Solution

CreditSeer reimagines credit agreement review with a source-grounded, analyst-ready workspace that surfaces critical terms and verifies them, accelerating review time by 50% without sacrificing control.

Smart Extraction

A proprietary LLM model extracts exact values scattered across the agreement and organizes them for analyst review.
Screenshot of CreditSeer's extraction panel showing values pulled from a credit agreementClose-up of an extracted value and its confidence indicator in CreditSeer

Source Verification

Document Viewer allows analysts to see the exact location of the extracted value in the document with one click.
Screenshot of CreditSeer's Document Viewer linking an extracted value to its source clause
*Go to more design decisions for a breakdown of tradeoffs

Human-in-loop

Analysts can correct values, which in turn trace back to whichever extraction stage failed, feeding directly into model refinement.
Screenshot of an analyst correcting an extracted value in CreditSeerScreenshot showing a corrected value tracing back to its extraction stage in CreditSeer
Impact

CreditSeer is live in pilot with an Atlanta-area regional bank. My work didn't stop there, it became infrastructure the lab still uses.

50%

Faster review for credit analysts, self-reported by 5 pilot analysts

Weeks → Days

Faster product validation cycles

Annotation Schema

Used to tag data and train the proprietary extraction model

Design System Components

Reused across lab projects

Domain Documentation

Used for team alignment and onboarding

Detailed process below ↓

This problem space wasn't just slow. It was expensive for banks.

Understanding business context

For regional banks, slow loan cycle means lost deals. And lost revenue.

To identify where the greatest friction was highest, I spoke with people across banking roles and mapped their workflows. Credit agreement analysis emerged as a recurring task across roles and throughout the loan lifecycle.

Workflow map of banking roles involved in loan review, from initial research
Grounding the design

I reviewed real agreements, interviewed analysts, studied LSTA standards, model constraints, and compliance requirements to ground how information should be surfaced.

Reference material reviewed while grounding the design: sample credit agreements and LSTA standards
Concept Explorations

Analysts needed traceable data, while leadership wanted portfolio-level insights, all within engineering and timeline constraints.

I used rapid prototypes and iterative conversations with all stakeholders to make these tradeoffs tangible, ultimately helping the team converge on an MVP direction

Early concept sketches exploring tradeoffs between analyst and leadership needs

The first prototype exposed a major product roadblock...

Early Usability Testing

I connected the UI and model early on in the process to test with real data rather than assumptions.
‍
But then the UI broke down. The proprietary model wasn’t yet reliable on real agreements, and the design system had no graceful way to handle its failures.

Screenshot of the early prototype UI breaking down during usability testing

Testing with dummy data wasn't an option; it wouldn't reflect how analysts actually work.

Designing a reliable experience meant waiting for the model to mature, costing valuable stakeholder access and user testing time.

So, I paused the UX work.

System design

I wanted to first understand what such LLM models can reliably extract from credit agreements.

So I built a lightweight extraction pipeline to design around those constraints rather than ideal outputs.

Preparation for the pipeline

Interviewed analysts on the values they need, then encoded patterns from 15+ agreements into an annotation schema.

Worked with ML engineers to design the annotation schema around how their model works.

Step 0

Full credit agreement PDF is converted into text

picture_as_pdf
Credit Agreement.pdf2.4 MB · PDF
Step 1

Agreement is split into section-aware chunks

Cover

CREDIT AGREEMENT — excerpt

$250,000,000 · REVOLVING CREDIT AGREEMENT

This CREDIT AGREEMENT is entered into dated as of March 15, 2024 among ACME HOLDINGS, INC., as Holdings, the Borrowers party hereto, and ROYAL BANK OF CANADA, as Administrative Agent…

Definitions

ARTICLE I — DEFINITIONS

“Applicable Margin” means, with respect to any Term Loan or Revolving Loan, the rate per annum set forth below…

“Effective Date” means the first date on which the conditions in Section 4.01 are satisfied…

Step 2

Model locates exact clause block, then extracts required value using hints from schema

Given to Stage 1
Given to Stage 2
Output
Definitions

ARTICLE I — DEFINITIONS

“Applicable Margin” means…

“Borrowing” means Loans of the same Class…

arrow_forward feed in
psychologyStage 1model
arrow_forward returns
§ Definitions · p.12 · located block

“Applicable Margin” means, with respect to any Term Loan, a rate per annum equal to 2.50%.

arrow_forward feed in
psychologyStage 2model
arrow_forward returns
{ "fieldId": "applicableMargin", "value": "2.50%", "section": "Definitions", "page": 12 }
data_object Annotation schema
description margin.stage1.json
1
2
3
4
5
{ "fieldId": "applicableMargin", "anchorPattern": "Applicable Margin", "task": "locate definition block" }
data_object Annotation schema
description margin.stage2.json
1
2
3
4
5
6
7
{ "fieldId": "applicableMargin", "extract": { "valueType": "percent", "pattern": "rate per annum equal to" } }
Step 3

System then presents the value in the UI as a card source-linked to the document viewer

description Document viewer

ARTICLE I — DEFINITIONS

“Affiliate” means, with respect to a specified Person, another Person that directly or indirectly Controls…

“Agreement” means this Credit Agreement, as amended, restated, supplemented or otherwise modified…

Credit Agreement.pdf · Definitions · p.12

“Applicable Margin” means, with respect to any Term Loan, a rate per annum equal to 2.50%.

“Borrowing” means Loans of the same Class and Type made, converted or continued on the same date…

Key Design Decisions

The pipeline revealed exactly where model could be reliable and where it was not. Now I could design for real outputs and not just the happy path.

Adaptive Layouts

Before: One component accommodated for all text sizes
After: Each value type maps to the component best suited to display it.
Before and after comparison of the component handling different value text sizes

Making Uncertainty Visible

Before: A single state treated every model output the same
After: Four states, confident, loading, ambiguous, and missing, make uncertainty explicit
Before and after comparison showing confident, loading, ambiguous, and missing states

Source Verification

Before: Unsure how to trace and show source
After: Direct way to trace value back to the source by Document View
  • Opens directly at the source clause without leaving the dashboard
  • Preserves the clause’s position within the agreement
  • Supports flexible multi-monitor workflows
A tradeoff I accepted AI complacency
‍
Because source review requires a click, analysts could still use extracted values without verifying them. I kept verification easy, but not mandatory for MVP.
‍
Future solutions:
  • Track behavior: Monitor when analysts check sources
  • Review / Audit modes: Values for fast scanning vs. value + source always visible
  • Copy to verify: Open the source when an analyst copies a value


Iterations that were rejected:
Rejected design iterations explored before converging on the final approach

The process got faster. The lab got a foundation to build on.

What I Learnt

This project changed how I think about product design in AI systems. I learned that designing the interface alone isn’t enough, you sometimes need to understand and shape the underlying system to create an experience you can actually trust.

Building with AI tools allowed me to develop even ML pipelines and learning the domain showed me how much context is required to simplify complex information well.

Most importantly, working with imperfect models reinforced the value of designing for uncertainty and getting access to real users and data early, especially in high-stakes workflows.

What I do differently and next

Looking back, I'd test feasibility before designing screens. Early testing showed the model's inconsistent outputs could break the interface, so next time I'd run a small extraction test on a handful of agreements first, and design screens only once I knew which fields were reliable.

I'd also keep
a standing loop with stakeholders and end users: in B2B work, the people who use the product and the people who buy it are different, and both need constant contact, so I'd set up recurring check-ins from the start rather than letting feedback stall between ad-hoc meetings.

What's next is
deeper integration into the loan lifecycle, extending beyond credit analysis to the other roles that touch the same agreement, like Relationship Manager, Legal, Credit Committee, and Loan Admin, along with governance layers such as role-based access and version tracking, building on the existing edit history and the system-generated versus user-modified distinction.