Fannie Mae’s Unique Approach to Evaluating the Value of Copilot
Fannie Mae’s Deployment of Microsoft Copilot
Over the past year, Fannie Mae has been introducing Microsoft Copilot to enhance productivity among its knowledge workers. Initially starting with a small group of 50 early adopters, the company has now expanded to a total of 500 users. Insights from a recent session at Enterprise Connect revealed the experiences and knowledge acquired from this ongoing deployment. With a significant investment of $30 per user monthly, determining the effectiveness and value of Copilot has become essential for the organization.
Understanding the Value of Copilot
Fannie Mae is at a critical juncture as they seek to clarify Copilot’s value to users and identify the next groups to receive access. James Redmore, Technology Director at Fannie Mae, emphasized the broader goal of addressing the company’s AI challenges, rather than solely focusing on Copilot. As they navigate these challenges, they are creating new tools and strategies to enhance their approach.
Benefits Experienced by Users
The phased rollout allowed Fannie Mae to deploy Copilot to various user groups, referred to as ‘personas.’ Users have reported significant benefits from key functionalities, including:
- Drafting Emails: Copilot assists users in quickly creating email drafts, allowing them to focus on refining their message.
- Revising Text: It helps in rewriting sentences, making communication clearer and more professional.
- Starting Documents: Users can enter basic ideas into Copilot, which then helps generate the initial draft of documents.
- Content Summarization: Particularly valuable for executives and researchers, Copilot helps summarize large amounts of information, saving them time and effort.
Training and User Support
Effective training has played a crucial role in the successful adoption of Copilot at Fannie Mae. The organization held several workshops to instruct users on how to leverage Copilot for their specific roles. Additional initiatives include:
- Weekly Tech Tips: Regular updates to provide insights on using Copilot effectively.
- Dedicated Support Hours: Specific times set aside for users to seek help with their Copilot queries.
- Center of Excellence: A community platform where employees can share knowledge and best practices.
- Best-Known Prompts Library: A collection of proven prompts designed to optimize the outcomes users can achieve with Copilot.
Real-World Application of Copilot
One prominent use case involved Fannie Mae’s single-family customer engagement team. This group was proactive in exploring how Copilot could enhance their workflows. During monthly customer calls, they utilized historical interaction data and crafted detailed prompts, resulting in more tailored engagement strategies. Rather than just generating text, this approach involved synthesizing a significant amount of information to create actionable insights.
Measuring Usage and Effectiveness
Despite several success stories, Fannie Mae discovered that not all license holders were actively using Copilot. Engagement levels varied widely across personas. To address this, they organized a workshop allowing users to self-assess how often they utilized Copilot and the time saved in various tasks. Redmore noted the need for a more structured feedback system due to the diverse range of responses.
Developing an Assessment Framework
The analysis framework introduced by Fannie Mae’s single-family analytics team allows for a systematic evaluation of AI tools, including Copilot. This two-hour workshop involves ten questions pertinent to the business, where different personas can answer some questions with AI assistance, while others must rely on their knowledge. This dual approach helps assess the potential value of AI tools accurately.
Future Steps in License Management
Looking ahead, Fannie Mae plans to implement a usage-based license recovery system for Copilot. This strategy aims to reclaim licenses from users who do not maximize the tool’s capabilities, making those licenses available to individuals who will benefit the most. By analyzing user engagement and aligning licenses with active users, Fannie Mae aims to manage AI tools as valuable operational assets rather than merely a technology expense.
