Business leaders have been hotly debating the advantages and drawbacks of remote work since COVID-19 forced a radical work-from-anywhere experiment. Some employers report improved employee satisfaction and reduced costs; others say productivity and teamwork have suffered.
A recent Harvard Business School study sheds light on how the quality of work in at least one field—auditing organizations’ compliance with management systems standards—fell during the pandemic-era switch from in-person audits to remote reviews. Understanding the problems auditors encountered could help improve remote work in the future, the authors say.
“There’s real quality loss potential that we need to be aware of and to think about when it comes to some remote work,” says Michael W. Toffel, the Senator John Heinz Professor of Environmental Management and one of the study’s authors. “Our findings don’t apply to every field. Some previous studies have shown there are contexts like call centers where there is no quality loss when doing things remotely versus in-person. But our findings do indicate a loss of quality loss when conducting monitoring and inspection work, at least in the years surrounding COVID-19 that we analyzed.”
Toffel and former HBS doctoral student Ashley Palmarozzo and former HBS research associate Melissa Ouellet wrote “Managing Remote Work Quality: Evidence from Auditing Management Systems Standards” after reviewing tens of thousands of global compliance audits before and after the pandemic’s outbreak in early 2020. The paper could inform other industries—from telemedicine to higher education—that are exploring how to optimize remote and hybrid work settings, they say.
Analyzing 35,000 audits
For their study, Toffel, Palmarozzo, and Ouellet effectively audited the auditors, evaluating nearly 35,000 compliance audits conducted in-person or remotely across thousands of global sites from 2019 to 2021.
The audits focused on compliance with six of the world’s most popular management systems standards, including ISO 9001 Quality Management Systems and ISO 14001 Environmental Management Systems. The work spanned sites in China, India, the US, the UK, and nearly 100 other countries.
The compliance audits entail gathering information via site tours, interviews, and document reviews. Before the pandemic, nearly all of these audits were conducted in person.
After the COVID-19 outbreak significantly limited travel, thousands of audits were conducted remotely via conference calls, video meetings, and document reviews, while thousands more continued to be conducted in person. Over the three-year sample period, 61 percent of the audits were conducted fully in-person, and the remaining 39 percent were handled remotely.
Slippery floors are hard to detect on Zoom
Remote audits detected 25 percent less violations on average than in-person audits, hinting that some violations that were observable in person may have been missed.
Functions that benefit from in-person observation, such as assessing whether production line conditions meet requirements and whether personal protective equipment is being used appropriately, had the most potential lapses. There was a 59 percent drop in violations of these types of standards’ clauses for which compliance is primarily detected through direct observation.
The major reason the authors hypothesized for the performance decline when switching from in-person to remote audits: Lack of access to the kind of sensory information that is hard to gather remotely.
In many cases, there was simply no substitute for auditors’ ability to physically see, hear, and feel what was around them when visiting sites. A smell at a factory might tip off auditors to a potential problem or impromptu conversations with on-site employees they might not have otherwise met might reveal unexpected deficiencies.
Document reviews seem like they could be handled equally well from afar, but even that task benefited from physical presence. For standards’ clauses for which compliance is primarily detected via document review, remote auditors found 21 percent less violations than in-person auditors.
Improving the quality of remote work
From a collaboration standpoint, the authors theorize and find evidence that the quality of remote audits suffers less when conducted by a single auditor than a team because it’s harder for the remote auditors to collaborate.
Moreover, the authors find that prior in-person auditing experience at the audited site matters, as the in-person site familiarity attenuates the disadvantage of remote auditing.
‘A hybrid solution’
There’s no one-size-fits-all fix to the thorny issues tied to remote work, the authors say. However, the study says that companies should tread carefully with functions or roles that require a person to use their senses.
“Given that many other work settings besides auditing—from telemedicine to drone-based crop monitoring to regulatory inspections—require direct observation, our results suggest that managers in those domains should take special care to control the quality of remote work,” the study concludes.
For many companies, a combination of in-person and remote work might ensure that employers get the highest quality work from employees.
“There's probably a hybrid solution here where some activities, or some parts of your job, can be done remotely without potential quality loss and then there’s some parts of your job that just have to be done in person,” Palmarozzo explains.
Image: Ariana Cohen-Halberstam with asset from AdobeStock/Kostiantyn
