Main Page > 14 Red Flags > ​​Audits > Red Flag 9

9. Audit Passed on a Non-representative Sample or Insufficient Sample Size

  • What this means 
  • Why investors should care
  • What to look for 
  • Checklist
<
>
Most audits as part of a certification scheme are carried out on only a small percentage of worksites, focus only on Tier 1 suppliers, or exclude upstream businesses, such as commodity growers.[1] A study conducted by researchers at The University of Sheffield documenting widespread human rights abuses in the cocoa industry reported that social certification schemes either rely on farmers self-reporting to cooperatives that they meet the certification standards, or audit only a small sample—sometimes as little as 5%—of the farms within a cooperative.[2] In other instances, whole segments of the supply chain may be left out. In the garment industry, for example, a factory may be audited but the cotton growers and textile workers may fall outside of the certification scheme. As a result, a company may tout certification or label its products even though a large share of its operations remains unaudited. 
 
Yet sampling is necessary, especially, in smallholder agriculture, where auditing every worker at every site is impractical - so a representative sample must suffice. Because vulnerable workers or community members are often reluctant to speak, a larger sample size logically improves the likelihood that some will. According to Nick Weatheril, Executive Director of the International Cocoa Initiative, commenting on the common practice of auditing selected cocoa farms once a year for child labor:
If you’re trying to use that light and occasional coverage to check for the occurrence of something that happens from one day to the next… you’re not really going to be picking up on the issue.[3]
​When an auditor understands site-specific risks and can detect hard-to-find problems (see Red Flag 7), over-sampling is warranted. In all cases, the scheme should define how it will sample different types of problems. 
[1] Genevieve LeBaron et al., Forced Labour Evidence Brief: Social Auditing and Ethical Certification (Re:Structure Lab, July 2022), 13, https://static1.squarespace.com/static/6055c0601c885456ba8c962a/t/62d746146f5dc5205a17621c/1658275349325/ReStructureLab_SocialAuditingandEthicalCertification_July2022.pdf.
[2] Genevieve LeBaron, The Global Business of Forced Labor: Report of Findings (University of Sheffield, 2018), 44-45, 
https://respect.international/wp-content/uploads/2018/06/The-Global-Business-of-Forced-Labour-Report-of-Findings-University-of-Sheffield-2018.pdf.
[3] Cited in Oliver Nieburg, “Fair Game: How Effective is Cocoa Certification,” Confectionary News, December 20, 2017,  https://www.confectionerynews.com/Article/2017/12/20/Fair-trade-How-effective-is-cocoa-certification.
When initiatives allow for only a small portion of a company’s operations to be audited at infrequent intervals, they can leave gaps in due diligence. As a result, risks such as child labor may remain hidden and when discovered by reporters or NGOs could expose the brand using on-product labels to allegations of deceptive marketing.
 
A lawsuit filed by Corporate Accountability Lab (CAL) in 2021 against the Hershey Company and Rainforest Alliance for deceptive marketing alleges that Rainforest Alliance only requires third-party inspections on “about 10 percent of the cocoa farms that companies such as Hershey source from,” asserting that this is one of the reasons the Rainforest Alliance’s certification system is “inherently inadequate.”[1] The year before the lawsuit, Rainforest Alliance introduced risk maps and began to require certified farms in medium- to high-risk areas to conduct in-depth risk assessments and collaborate with stakeholders (see “What to Look for”).
​
→ Demonstrates: Reputational risk, legal risk
[1] Corporate Accountability Lab v. The Hershey Company: Complaint Demand for Jury Trial, Superior Court of the District of Columbia (2021), 14-15, 
https://static1.squarespace.com/static/5810dda3e3df28ce37b58357/t/618167d28dd7f307c90da0e4/1635870679248/CAL+v.+Hersheys+and+RA_Stamped+Complaint.pdf.
Since it is typically impractical to interview all workers or community members or audit all worksites in a complex supply chain, schemes should take a human rights risk-based approach and provide guidance to companies on how to sample multisite entities. In addition, investors can ask companies and schemes what they are doing to improve their audit sample size and coverage. For example

  • Does the scheme provide clear guidance on the sampling methodology, such as the percentage and categories of affected stakeholders that are interviewed at each site and a guarantee that all ethnic groups be covered? (See Red Flag 4)
  • Does the scheme require companies to provide a rationale for the sampling methodology?
  • Does the scheme require adhering entities to procure an increasingly larger percentage from the certified entity over time?

For initiatives that have a supplier standard,
  • Does the scheme require adhering entities to audit all of their direct suppliers and most salient supply chain risks? (See Red Flag 1)
  • Does the scheme provide a way for companies to determine the percentage of their direct sourcing that the standard or certification represents, and does it require the company to publicly disclose this information?
  • Considering the rapid advances in social tracing technology, does the scheme use or encourage companies to use tracing technology?[1]
 
The following certification schemes provide a clearly stated sampling methodology that companies should follow:
➔    The Aluminum Stewardship Initiative (ASI) takes a risk-based approach to assurance. Its Assurance Manual states:
[I]deally all Facilities in a Certification Scope should be visited and this should be the aim over a reasonable time period. However, where an Entity’s activities within the Certification Scope are carried out in a similar manner at different Facilities, and that all are managed and controlled by the Entity’s systems and procedures, a representative sample of sites can be selected.[2]
ASI only allows for representative samples when all sites conduct similar activities, use comparable equipment, or are managed under the same head office.

➔    The Rainforest Alliance 2020 Sustainable Agriculture Certification Program requires that for an ingredient to be certified, all actors along the supply chain must meet the certification requirements, which are published online. Audit sampling is risk-based, meaning that “risk analyses are carried out at the farm and supply chain level and data is submitted via the Rainforest Alliance Certification Platform at several different stages of the certification process. This data is used to identify the relevant requirements for each farmer or company.”[3]

➔    Fairtrade International has a Trader Standard that applies to the entire supply chain. Specifically, the standard specifies that it
applies to all who trade (buy, sell or process) Fairtrade certified products, up to the point where the product is in its final packaging. In other words this means that within this scope all operators who take legal ownership of Fairtrade certified products must be audited and certified against these standards.[4]
​Additionally, the standard requires all traders be audited to ensure compliance with the standard, applicable environmental and labor law, and the ILO Core Convention.[5]
[1] See for example, “STREAMS,” Verité, accessed July 9, 2024, https://verite.org/streams/.
[2] Aluminium Stewardship Initiative, ASI Assurance Manual version 2 (May 2022), https://aluminium-stewardship.org/wp-content/uploads/2022/05/ASI-Assurance-Manual-V2-May2022-3.pdf.
[3] Rainforest Alliance, What's in our 2020 Certification Program? Risk-Based Assurance and Contextualization (June 2020), 
​https://www.rainforest-alliance.org/wp-content/uploads/2020/07/whats-in-our-2020-certification-program-risk-based-assurance.pdf.
[4] Fairtrade International, Explanatory Document for the Fairtrade Trader Standard (August 2015), 4,  https://files.fairtrade.net/standards/2015-09-04_Explan_Doc_GTS_EN.pdf.
[5] Flocert, Trader Certification Compliance Criteria, 66, June 28, 2024. https://www.flocert.net/fairtrade-compliance-criteria/.
Does the scheme provide requirements and/or clear guidance for member companies’ sampling methodology?​
​❐  Yes 
❐  No
❐  Partially

Are member companies required to procure a larger percentage of their supply product from certified suppliers over time?
❐  Yes 
❐  No
❐  Partially
This project is a collaboration among the following organizations:
 This site is maintained by Rights CoLab. 
Copyright © 2024