The Law of Unintended Consequence – A Didactic Tale for Law Firms
We are strong advocates for the effective adoption of AI and Generative AI at scale within legal departments and law firms. However, as with any new technology, leadership must consider the possible permutations – both commercial and human – when implementing AI and Generative AI. These new technologies change how we work, the type of work we can or should be doing, how much work we can do, and how we price the work we do. This makes their implementation arguably more consequential to firm operations and people than any technology that has come before. Therefore, it should be carefully and thoughtfully implemented.
In the first edition of our Leadership Publication, we suggested:
“Global and international law firms may need to reconsider their commercial models to ensure the best blend of technology-enhanced expertise and cost-efficiency. They should also be alert to the tension between their offices in low-cost and high-cost jurisdictions. With large language models (LLMs) levelling the playing field, competition between different offices for the same clients and work may need careful regulation. Clients are becoming increasingly adept at leveraging international networks to secure both expertise and cost advantages. Global firms should act swiftly to provide necessary commercial guidelines across all international locations. This may be more complicated in firms that are not financially integrated, in which case some form of enhanced revenue-sharing model may be needed.”
Since then, the impact of Generative AI has become increasingly apparent. Productivity gains of 20-30% in the form of improved efficiency in first drafts, research, and similar tasks are now commonplace. The positive argument for these gains is that the time freed up is now available for additional higher-value work. But what if that work is not available or your partners are not aggressively seeking it? What is the impact of this new efficiency if your ways of working are disrupted without any new additional deal flow? Is there any human cost to the new model?
We were recently asked to advise an international firm on the unanticipated tension between two offices following a firm-wide lifting of an embargo on the use of an LLM. The firm’s approach was to “be innovative” and leave it to the partners and lawyers to see what use they could make of the LLM. An AI policy was published covering ethical use, client confidentiality, and data security. However, what was not considered, except at a firm “headline” level, were the possible operational, commercial, and most importantly, human consequences of using the LLM by different partners, lawyers, teams, and offices.
Case Study – What Could Go Wrong
Using its international footprint, the firm had for several years leveraged the commercial advantage of resources in lower-cost offices to assist on matters originated in higher-cost offices, all with client approval. In one sector, the firm had created an excellent workflow powered scheme, allowing an originating partner and his legal project manager to allocate work across two country offices, ensuring that the appropriate expertise in the best time zone was used at every step. This ensured effective management of resources and costs, providing both a beneficial matter rate to clients and maintaining a constant margin on the matters worked.
Included in the final bill to clients was a 15% premium on the hours worked by the lawyers in the low-cost office. In the internal accounting, this premium was allocated to the originating partner. The premium made the arrangement attractive to lawyers in both offices since the lawyers in the low-cost office received 100% of their rate and the originating partner received the credit of the 15% premium charged on the lower rate, effectively incentivising him to push work to his colleagues in the low-cost office. Most importantly, clients received great service – faster and cheaper than if it had been done only by the originating partner’s team.
For a group of four sector clients, the firm had a team of 25 lawyers working on this scheme: 3 in the high-cost office and 22 in the low-cost office. Given the importance of the clients and the volume of matters, the team of 22 in the low-cost office, comprising one partner, 12 associates, and 9 paralegals, used on average 90% of their capacity on the scheme. They were measured and incentivised based on the scheme’s success. Meanwhile, the originating partner was measured and incentivised on his wider practice, primarily on revenue, profit, and hours billed by him and his team.
By the time the LLM embargo was lifted, firm lawyers and paralegals had already been exploring the possible use of LLMs in their own time. The tech-minded and innovative among them had been using publicly available LLMs in their private lives and quickly became adept at using these on public platforms with public data. Many had been eager to use the technology at work, arguing it would make their lives easier and them more efficient.
Within weeks of the LLM embargo being lifted, a paralegal and associate in the originating partner’s team demonstrated that the LLM could deliver roughly 50% of the work previously outsourced to the low-cost office – in minutes rather than hours – and that the supervision of the LLM output could easily and quickly be done by them. Seeing the efficiency gain and the perceived benefit of controlling more of the work locally, the originating partner adopted an LLM-enhanced version of the scheme with little discussion with his colleagues in the low-cost office and without fully considering the commercial and human impact.
The Impacts – A Double-Edged Sword
Positive Impacts
By adopting the LLM-enhanced scheme, the originating partner reduced the fee charged to the scheme clients by nearly 10%. They were delighted – and the reduced fee became the new benchmark.
The originating partner team became adept at using the LLM to improve team efficiency and productivity. More opportunities to adopt the LLM were identified and actioned.
The originating partner’s performance metrics improved by the addition of the new hours now recorded by his team on vetting the scheme LLM output.
Negative Impacts
On the firm side, the LLM-enhanced scheme reduced the revenue recorded by the low-cost office and the firm overall, retained the same cost base, and reduced the overall profit on the scheme by almost 16%. Although his team retained more of the available hours, the profit generated by the originating partner’s team decreased – largely due to the reduced premium on the low-cost hours – negatively impacting his team’s P & L statement.
On the human side, the sudden adoption of the LLM-enhanced scheme was devastating to the low-cost office team. Almost overnight, they found themselves with a capacity glut, behind budget on hours and revenue, and unable to meet the performance and incentive metrics agreed with the firm. With performance appraisals looming, tempers frayed, and relationships soured. Management was accused of playing favourites, and partners in the low-cost office perceived themselves and their colleagues as second-class citizens. What caused the most danger is that management quickly found its credibility under scrutiny: Why was this issue not considered before making the LLM available? Why was there no communication or proper training? What financial modelling had been done on the potential effect of adopting the LLM? How would performance management, compensation, and incentive schemes be impacted? What impact would the new technology have on career progression?
So, what should have been a positive brand, commercial and talent retention story quickly soured. On the bright side, the clients were more than happy with their unexpected windfall.
The Moral of the Story – Lessons for Law Firm Partners and Leaders
- Before implementing new technologies, thoroughly consider their potential impact on all aspects of the firm, including financial, operational, and human factors.
- Reevaluate and adjust the organisational structure to ensure that technology-augmented tasks are assigned to individuals with the right skills and cost profile. While giving everyone unrestricted access to efficiency-improving technology might seem logical, it can lead to unexpected and disruptive outcomes if not carefully managed. For instance, critical tasks might be mishandled, leading to inefficiencies and errors.
- Additionally, there are potential negative commercial consequences, such as having an LLM prompted and supervised by expensive lawyers in a high-cost jurisdiction when equally skilled, lower-cost lawyers in another location could perform the task more cost-effectively, thereby better protecting margins.
- Ensure clear and continuous communication with all stakeholders. Explain the reasons for changes, the expected benefits, and the potential challenges.
- Involve representatives from all affected areas in the decision-making process to gather diverse perspectives and foster a sense of ownership and cooperation.
- Provide comprehensive training and support to all employees to help them adapt to new technologies and workflows. This includes upskilling in both high-cost and low-cost jurisdictions.
- Reevaluate and adjust performance metrics, incentives, and compensation schemes to reflect the new reality brought about by technological advancements.
- Continuously monitor the impact of new technologies and be prepared to adjust as needed. Flexibility and responsiveness are key to managing change effectively and, in the instance of technology like AI and gen AI, making sure that the technology delivers a commercial benefit.
These points may sound self-evident, but they require particular focus given the far-reaching commercial and human impact of the new technology. By taking these lessons to heart, law firm partners and leaders can better navigate technological integration, ensuring that the benefits are maximised while minimising any negative consequences. This story serves as a reminder of the importance of foresight, communication, organisational design, and inclusive leadership in the rapidly disrupting legal landscape.