Congress is Investing in Custom AI. Here are the Questions that Need to be Asked.
BY ASHLEY NAGEL & MARCI HARRIS
The House of Representatives is debating investing in AI systems tailored to the Legislative branch. If successful, these initiatives would put the House among leading legislatures in beginning to harness AI for institution-specific use cases to increase capacity.
House Modernization and Innovation Subcommittee Chairwoman Stephanie Bice [R, OK] formally requested funding in July for My Intelligent Assistant, or Mia, described as “a secure and customizable AI agent” connected to House data sources. She also announced funding for PRISM, a replacement for the Rules Committee’s legacy legislative-management system.
Meanwhile, Acting Librarian of Congress Robert Newlen told Senate appropriators this spring that the Library needs $5.4 million to develop a robust AI platform so that the Library and Congress will not “be left behind.”
These initiatives show promise that the House and other entities across the Legislative branch recognize the need for institutional AI. The announcement of these projects also raises a set of practical questions that Congress will need to answer before choosing any particular model or vendor.
What should power Mia?
Mia appears well suited to be built with models and retrieval-augmented generation, or RAG, system. A RAG system searches approved institutional sources and provides relevant material to an AI model when a user asks a question (for more on RAG, see POPVOX Foundation’s What Is RAG And Why It Matters for Legislative AI Use Cases). This would allow Mia to draw from legislative data, House rules, office policies, and other authoritative sources without expending the effort required to encode all of that information permanently into a model.
POPVOX Foundation demonstrated the feasibility of this approach last year with its StaffLink bot prototype.
To guarantee the successful rollout of this platform, the next questions the House (or other legislative institutions pursuing similar initiatives) must answer are:
Which model or models will generate Mia’s answers?
Will House information be sent to a commercial provider, or will the model run in a House-controlled environment?
Can the underlying model be replaced without rebuilding Mia?
How will the House evaluate accuracy and receive feedback from users to improve the system?
Should the system preserve auditable logs identifying which model and set of sources produced a particular answer?
What is the Library of Congress proposing to build?
The Library is proposing a similar but much more significant undertaking, and an open question from their testimony is how much they intend to build from scratch. Will the Library pursue a platform based off of commercially available models, a locally-hosted open-weight model, or a model that the Library trains using its own data?
These are very different undertakings. If the Library decides to go the commercial route, it could still offer access to multiple models while maintaining common security, data-governance, and evaluation standards. If it intends to customize an open-weight model, it could provide more tailored experiences, but at the cost of assuming a much larger and more permanent technical responsibility.
Before taking that step, the Library should evaluate what fine-tuning an open-weight model would accomplish, as opposed to a closed-weight model with well-designed system prompting and RAG. Institutional knowledge that changes frequently is generally better retrieved from authoritative sources than embedded in a model’s parameters. In fact, a common misconception is that fine-tuning reduces hallucination. In practice, fine-tuning on office documents may teach the model the correct form or style of an answer, but not the accuracy of its content.
If the Library decides to customize an open-weight model, it must first also address the following questions:
Which base model will it select, and according to what evaluations?
Where will it be securely hosted?
Will the Library be able to easily transfer its training data, evaluations, and adaptations to another model?
Does it have the continuing staff and computing resources required to maintain the system — not merely the money to launch it?
Which models are realistic options?
Most readers will recognize popular closed-weight models: OpenAI's GPT, Anthropic's Claude, and Google's Gemini, along with products built on them such as Microsoft Copilot. Each can be offered as an enterprise-wide service, and each carries pros and cons the House and the Library will need to weigh.
Open-weight models are likely less familiar. Examples include Meta's Llama and Glimmer, Google's Gemma, OpenAI's gpt-oss, and Thinking Machines' Inkling. For more on recent open-weight model developments, see POPVOX Foundation’s AI Model Customization: Why It Matters for Policy Makers.
For elected officials, staff, and public servants trying to navigate these distinctions, POPVOX Foundation has also developed Open, Closed, and Customizable: A Primer on Model Selection. It explains terms such as open-source, open-weight, and closed models, as well as the differences among system prompting, RAG, fine-tuning, and training from scratch.
For the House’s and Library’s purposes, open-weight models should be evaluated alongside commercially available closed models, but not assumed to be preferable simply because their weights are available, allowing for the possibility of fine-tuning. The relevant questions are whether they perform well on Congressional tasks, can meet security and reliability requirements, and can be maintained at a reasonable cost.
Note: Many of the most high-performing open-weight models, including the recently released Kimi K3, originate from China. Although Congress does not yet appear to have a categorical rule prohibiting the use of Chinese-developed models, the political, security, supply-chain, and public-trust concerns would make such a choice a practical nonstarter.
Institutional AI: Investing in Congress’ Data Infrastructure
Congress may ultimately find that no single model is best for every purpose. A secure commercial model might be appropriate for general use, while a locally-hosted American open-weight model is more suitable for handling sensitive information entirely on premise, or high-volume work that could incur excessive costs on a commercial platform.
Congressional organizations should be very wary of designing AI platforms around a single model. Capabilities, prices, security characteristics, and licensing terms are changing too quickly. A model that’s evaluated to be the best option during procurement may no longer be so by the time the system is deployed. Instead, the most durable public investment legislatures can make in this AI era is in the infrastructure to support AI models, not in the choice of model specifically.
The institution should focus on building authoritative legislative data collections, access controls, performance evaluations, audit mechanisms, and model-independent user interfaces alongside continued pilot projects to experiment with different AI tools and improve AI literacy across the Legislative branch. Those assets will remain useful even as the underlying models change. We call this “institutional AI,” as distinct from consumer AI for everyday users or enterprise AI for businesses.
This approach also future-proofs it by avoiding vendor lock-in. Congress could evaluate new models as they become available, switch providers when necessary, and compare outputs across models without continually rebuilding the entire system. Any fine-tuning work should, if possible, avoid creating a permanent dependency on one model family.
As Congress begins to seriously invest in AI, these early decision points are critically important. Congress needs a strategy that keeps pace with rapid technological change – not one that risks hamstringing itself later.
