Regulation Assistant

Background & Challenge

There are many regulations defining what can be done and what can’t be done in a real estate development. However, those regulations are different state by state, and they get updated over years. So even for those experienced designers, it is a challenge to make sure their designs are 100% compliant. It is also a constantly time & cost consuming exercise for a company.

Solution & Impacts

With the help of copilot backed with fine-tuned local deployed AI, the regulation compliance QA&QC cost got decreased by 64%, DOB objection rate got decreased by 23%. And those improvement is still ongoing as the local deployed AI constantly gets trained with latest data.

Team & Role

- Product Manager
- Product Designer (me)
- Product Prototyper (me)
- AI/ML Engineer
- Software Engineer

Impacts

Regulation compliance QA&QC cost got decreased:

64%

Regulator objection rate got decreased:

23%

The problem

Building Code, Zoning Resolution, American with Disabilities Act, LEED, WELL, Occupational Safety and Health Administration, American National Standards, etc. those are regulations governing every aspect of real estate development. However, each of them is inches thick book, furthermore, they get updated every several years. Every time new versions of regulation are released, it takes time or even several projects to completely digest them, lawmakers are also invited to design companies to give presentations and answer questions, this educational service are very expensive, but not equally effective some times.

As LLMs iterate in recent years, it looks like this is a perfect scenario that AI kicks in. So is AI the answer?

Research and findings

To deeply understand the problem problem, we started with reviewing the workflow of design professionals.

By analyzing the user journey map, there are four main pain points got identified:‍

Pain Point 1

Each regulation has many provisions, hard to read through them all and find out which provisions apply to the case.

Pain Point 2

Some regulation language are vague, hard to sort out the real requirement from long phrased provisions, some times it gets overlooked.

Pain Point 3

Provisions sometimes are not strictly objective, same term might have different interpretations in different context.

Pain Point 4

When articulating design with stakeholders, it’s hard to verify the regulation compliances of design quickly in real time.

Usually, those tasks need good amount of effort and time to process huge amount of content, thus those pain points can hardly be solved by either human themselves or human programed workflow. However, they are exactly what AI is good at, especially LLMs.

What is AI good at

Obviously AI has potential to solve all four pain points, however, the way AI processes jobs is completely different from human being. So in order to leverage its power, we need to make sure it has the capability to solve problems we expect it to solve, meanwhile, we need to make sure it understands the tasks users hand over.

If we translate this two subjects into design goals, they are:

Design goals

1. Build an LLM that specialized in Regulations.
2. Allow users to communicate with this LLM in an efficient way.

Solution for the design goal #1:

1. Instruction Fine-tuning & RAG

Instruction fine-tuning is to teach the LLM think like a design professional, the target is to make LLM reason like a regulation expert. The instruction fine-tuning can:
1. Shape the style of answers (i.e. cite sections, be precise, structured output);
2. Understand common question patterns in this domain;
3. Gain the ability to interpret ambiguous user requests;

Structured Dataset

And of course the fine-tuning has its limitation: it does not update the model’s raw factual database with every single regulation provision — that would require huge amounts of data and compute.

This is where we need to intergrade another specialization approach, RAG

2. RAG

RAG feeds the LLM the latest facts. The target of adopting RAG is to give the LLM correct regulation book it can refer to every time it answers. RAG is able to:
1. Allow LLM to directly access to full regulation text.
2. Pull relevant clauses for each query.
3. Update regulations without retrain the LLM.

How RAG integrated into user flow

The RAG has to be used together with fine-tuning in this case, otherwise the LLM doesn’t know how to properly use retrieved text. And only if these two approaches together, the LLM is able to give accuracy, adaptability and natural reasoning.

Response comparison before and after fine-tuning & RAG

Solution for the design goal #2:

1. Prompt engineering

Regulation interpretation can be complex, it depends on the project location, jurisdiction, regulation year, building category, construction type, etc. A broad prompt can lead to a completely wrong direction. Thus, prompt engineering is something critical to receive precise and applicable response.

How prompt should be properly templatized

2. Intuitive user interface

The User interface is designed to feel natural, effortless, and context-aware communication with the AI copilot. The target is to adapts to both human behavior—anticipating needs AI anticipating input, reducing friction, and guiding interactions seamlessly.

Key qualities criteria are:

‍Simplicity: Minimalist layouts that highlight essential actions without clutter.
‍Consistency: Predictable patterns that make navigation and interactions second nature.
‍Feedback: Immediate, clear responses to confirm that the assistant has understood and is acting.
‍Context-awareness: Guide users through, so they intuitively provide critical context data, so the prompt can be properly templatized.

Ultimately, the AI assistant UI should feel less like “talking to AI” and more like collaborating with a capable partner, who users tend to trust naturally.

Copilot interface

AI Regulation Assistant

Result