Hedge Fund Operational Due Diligence :
The Madoff Effect - An analysis of operational due diligence trends
Abstract
When Bernard Madoff turned himself in to authorities on December 11, 2008 hedge fund investors took pause to reassess their approach towards due diligence. Many of the red flags that were present in the Madoff Ponzi scheme, as well as other recent hedge fund frauds, were operational in nature. Indeed recent failures of many legitimate hedge funds, where no fraud was present, can be attributed to inherent and unmitigated operational risks.
As a result, many individual and institutional hedge fund investors, including fund of hedge funds, utilized the Madoff fraud as an opportunity to refocus their attention towards operational due diligence. Operational due diligence on hedge funds primarily focuses on diagnosing, analyzing and monitoring the operational or non-investment related risks of a hedge fund manager.
With this renewed and increased focus on operational due diligence, hedge funds reacted with a myriad of responses. Some concentrated on addressing, in due diligence questionnaires and marketing pitchbooks, what they thought would be investors key operational risk concerns. Others, hoping to placate investor concerns, placed their faith in purchased operational risk certifications. A few hedge funds have even pursued traditional manufacturing certifications, normally held by administrators, such as the SAS 70 certification.
This mixed bag of responses by hedge funds has further fostered a lack of uniformity in the type and scope of factors analyzed by investors during the operational due diligence process. As a result, many investors remain unsure which factors are appropriate or important to cover during the operational due diligence process. Making problems worse, those performing operational due diligence have had suffered from a dearth of benchmark data whereby they could effectively determine:
This study attempts to answer these and other questions by analyzing operational due diligence data collected from over 200 hedge fund allocation organizations.
The data shows that in anticipation of stricter hedge fund regulation, investors are focusing the bulk of their due diligence efforts on legal, compliance and regulatory risks. Interestingly, the data also demonstrates a so-called Madoff Effect . Analysis shows that frauds, such as Madoff, have caused investors to tailor their hedge fund operational due diligence process to focus on the red flags of recent frauds. While this approach has advantages, it assumes that fraud can be modeled and has led to marginalization of other equally important operational risk factors.
Introduction
A wide variety of approaches are taken by hedge fund allocation organizations that perform operational due diligence reviews of underlying hedge fund managers. Previous research has focused on analyzing the types of operational due diligence frameworks utilized by fund of hedge funds. Fund of hedge fund operational due diligence frameworks can be classified into four distinct buckets: dedicated, shared, modular and hybrid. This paper focuses on providing an analysis of the number and types of operational factors covered during an operational due diligence review. Brief definitions of each of these frameworks are outlined below for reference:
Dedicated - An operational due diligence framework where a fund of hedge funds has at least one employee whose full time
responsibility is vetting the operational risks at hedge fund managers.
Shared - An operational due diligence framework where the responsibility for reviewing the operational risk exposures at
hedge funds is shared by the same individuals who have responsibility for investment due diligence. No full time dedicated operational due diligence staff
are employed.
Modular - An operational due diligence framework whereby the operational due diligence process is classified into functional
components and parsed out among different specialists with relevant domain specific knowledge.
Hybrid - A hybrid operational due diligence framework refers to an approach that encompasses some combination of the
three previously described approaches (dedicated, shared, modular) and may incorporate the use of consultants.
Hedge fund allocation organizations can utilize the data in this study to benchmark the comprehensiveness and scope of their operational due diligence processes. Similarly, investors who allocate capital to pooled hedge fund allocators, including fund of hedge funds can utilize the data in this study to evaluate the scope and comprehensiveness of their respective allocator’s operational due diligence process.
Organizations That Were Included In This Study
For the purposes of this study, the term fund of hedge funds was defined as an investment organization whose primary purpose is to allocate capital to a portfolio of underlying hedge fund managers. The types of institutions from which data was obtained included those organizations that market themselves as fund of hedge funds, as well as multi-family offices, endowments, foundations, corporate pensions, large independent financial advisory practices and private banking organizations that manage portfolios of hedge funds on behalf of their clients. The hedge fund managers allocated to by the fund of hedge funds could be of any investment strategy. Fund of hedge funds that primarily allocated to funds that were not hedge funds (i.e. - private equity funds, venture capital funds, real estate funds etc.) were not included in this study. Finally, a globally diverse cross-section of fund of hedge funds was included in this study as there were no geographic restrictions.
Number of Fund of Hedge Funds Included in This Study
In order to respect the confidentiality of both those organizations that directly participated in this study or upon which research has been performed, the names of specific organizations have intentionally not been disclosed. Furthermore, the exact number of fund of hedge funds managers utilized in this survey is not specifically disclosed. This is to prevent reverse engineering of the specific identity of any one fund of hedge funds organization included in this survey. A range of the number of fund of hedge funds reviewed has been provided. In this study between 200 and 235 fund of hedge funds were utilized.
Data Sources
The data set utilized for this study was pieced together from several different sources. The primary sources utilized included interviews and surveys with employees working at funds of hedge funds organizations as well as other organizations, as described above, which allocate to hedge funds. Other data was collected from publicly available databases, websites and regulatory archives and via independent research. As a general comment, all percentages stated in this report have been rounded to the nearest whole percent.
Operational Factor Classifications
An analysis was performed by first attempting to broadly determine the number of operational factors analyzed by fund of hedge funds during their operational due diligence processes. This analysis was performed regardless of the operational due diligence framework employed. To facilitate this process the concept of operational factor classification was utilized. For the purposes of this study, the term operational factor refers to umbrella categories, which group several related sub-factors. It should be noted that the goal of this stage of the analysis was to determine how many factors, regardless of the type of factor, were covered during the operational due diligence process.
Within the operational due diligence field, practitioners commonly group several related data points under larger umbrella operational factor categories. Such classifications tend to facilitate ease of analysis and data grouping. However, universal operational factor category definitions are not general employed among different fund of hedge funds. So for example, while one fund of hedge funds organization may give the title of “ Legal and Compliance ” to one operational factor category, another organization can bifurcate this into two distinct factors such as, “ Legal Documentation ” and “ Compliance and Regulatory. ”
Consequently, to facilitate a homogenous comparison of operational factors across a level playing field we have to utilize a core set of operational factor elements. The core set of factors utilized for this study are outlined in Exhibit 1.
Exhibit 1: Core Set of Operational Risk Factor Categories
| Factor Categories | |
| 1 | Technology and systems |
| 2 | Business continuity and disaster recovery |
| 3 | Legal/compliance |
| 4 | Regulatory |
| 5 | Assets and investors concentration |
| 6 | Quality and length of relationship with service providers |
| 7 | Reputation of employees (including senior mgmt.) and firm |
| 8 | Valuation techniques and pricing sources |
| 9 | Compensation and employee turnover |
| 10 | Firm stability including expense analysis |
| 11 | Operations connectivity |
| 12 | Trade life cycle |
| 13 | Fund terms |
| 14 | Transparency and reporting |
| 15 | Counterparty oversight |
| 16 | Cash controls and management |
| 17 | Insurance |
| 18 | Independence and oversight of the board of directors |
| 19 | Other factors |
The “ Other factors ” category in Exhibit 1 serves as a catch-all to encompass other factors reviewed by fund of hedge funds which were not included in the eighteen core factors. During this stage of the analysis emphasis was placed on solely determining the magnitude (i.e. - number) of operational risk factors analyzed during the fund of hedge fund operational due diligence process. This stage of analysis was factor neutral. As such no biases were inherent against any particular operational factors and all factors were included, regardless of the identity of each actual operational factor.
Additionally, it should be noted that the operational factors in Exhibit 1 are organized by the operational areas covered within the category, rather than the way in which the factors in each category are analyzed. So for example, within the category entitled, “Firm stability includes expense analysis” different fund of hedge funds that include a review of this operational factor in the course of their operational due diligence process may, in practice, approach this analysis in different ways. Continuing our example, one fund of hedge funds may include in their analysis of this operational factor a review of historical audited financial statements for a particular fund vehicle, while other may not.
Furthermore, within those fund of hedge funds that did include financial statement analysis in their operational due diligence approaches multiple series of sub-set analyses may be present as well. These sub-sets of analyses can include length of financial statement history reviewed (i.e. – since inception or within the past five years), related vehicles (i.e. – if an investment is being considered in a feeder fund are the master fund’s financial statements reviewed) and unrelated entities (i.e. – if a hedge fund manages considered for direct investment are other the audited financial statements of these vehicles reviewed?). A detailed analysis of such nuanced sub-sets was not incorporated in this stage of the analysis. However, it is useful for the reader to consider the factor neutral empirical results set against the sub-set backdrop in order to facilitate a better understanding of the analysis.
Empirical Results: Framework Implication for Factor Analysis
The data demonstrates that across all study participants, regardless of the operational due diligence framework utilized, an average of approximately eight operational factors were analyzed. Within particular frameworks, the number of factors varied in line with expected predictions. For example, fund of hedge funds utilizing dedicated frameworks, on average, analyzed six more operational factors than the framework neutral results, for an average total of 14 operational risk factors. Modular frameworks tied with the framework neutral average, at eight operational risk factors. Hybrid frameworks fell just below the framework neutral average with seven operational risk factors analyzed. Finally shared frameworks, analyzed the least number of operational risk factors at five which was three less than the framework neutral average. These results are summarized in Exhibit 2.
Exhibit 2: Number of operational factors analyzed
| Framework | Average number of Factors of Within Each Framework * |
| Across all frameworks | 8 |
| Hybrid | 7 |
| Dedicated | 14 |
| Modular | 8 |
| Shared | 5 |
* Number of factors and percentages have been rounded to the nearest whole integer
Factor Commonality
The next set of the analysis was focused on determining the frequency of those factors which were reviewed during the fund of hedge fund operational due diligence process.
The data demonstrates that as a percentage of all the core operational factors outlined, including other factors in Exhibit 1 above, across all operational due diligence frameworks, “ Independence and oversight of the board of directors” and “ Technology and systems” were the least popular categories reviewed by fund of hedge funds at 1.18% and 1.76% respectively. This can be compared with “Fund terms” and “ Legal/compliance ” which were the most popular category reviewed at 9.65% and 9.84% respectively. This bias towards “ Legal/compliance ” confirms predictions that of enhanced investor sensitivity towards regulatory and compliance related matters in light of enhanced enforcement action by regulatory agencies, such as the United States Securities and Exchange Commission and the soon to be abolished United Kingdom’s Financial Services Authority. Likely also contributing to enhanced attention towards reviews of legal and compliance related factors is the evolving nature of legislation to further regulate hedge funds. These results are summarized in Exhibit 3.
Exhibit 3: Operational factors analyzed by fund of hedge funds
| Factor Categories | Factors Analyzed |
| Legal/compliance | 9.84% |
| Fund terms | 9.65% |
| Assets and investors concentration | 9.41% |
| Valuation techniques and pricing sources | 8.24% |
| Transparency and reporting | 8.01% |
| Trade life cycle | 7.06% |
| Regulatory | 7.06% |
| Firm stability including expense analysis | 6.48% |
| Quality and length of relationship with service providers | 8.01% |
| Business continuity and disaster recovery | 5.46% |
| Cash controls and management | 8.01% |
| Insurance | 2.94% |
| Other factors | 2.83% |
| Counterparty oversight | 2.46% |
| Compensation and employee turnover | 2.35% |
| Operations connectivity | 2.24% |
| Reputation of employees (including senior mgmt.) and firm | 1.76% |
| Technology and systems | 1.76% |
| Independence and oversight of the board of directors | 1.18% |
Operational Risk Factor Process Inclusion Does Not Imply Review Quality or Scope
In regards to the factors and associated percentages included in Exhibits 1 and 3, it should be noted that simply because a fund of hedge fund incorporated an operational risk factor into their operational due diligence process, this does not imply that the fund of hedge funds appropriately vetted this particular operational risk category. For example, consider the extreme example of a hedge fund manager who manufactures fraudulent financial statements from a phony auditor. Next consider two different fund of hedge funds, both of which collect and review these financial statements. Continuing our example, assume that as part of this review process these fund of hedge funds will consider both the quality and the hedge fund’s relationship with the auditor, who they do not know is phony. They therefore will have both satisfied the required of factor category requirement, outline in Exhibit 3 above, of vetting quality and length of relationship with service providers. The two different fund of hedge funds however, may come to different conclusions regarding this auditor (i.e. – one may be convinced the auditor is phony, while another is comfortable with the auditor based on a cursory review of the financial statements.)
Facilitating the conclusions of the two fund of hedge funds in our example, may be the robustness of sub-factor review, such as the number of years of financial statements reviewed, as well as the analysis of the financial statements of affiliated fund entities. As such, fund of hedge funds can differ not only in the quality of due diligence performed for each operational risk factor category, but in the scope of these reviews as well. In summary, this example, illustrates that simply because a fund of hedge funds reviews certain operational risk factors, they are not necessarily skilled or competent at uncovering fraud or fully vetting a particular operational risk.
Factor Weights and Sub-Factor Scope Aggregation
Among fund of hedge funds, the qualitative data suggested that a general perception persisted that certain factors carry more weight in the process of coming to an operational determination regarding the operational risk of a hedge fund manager. It should be noted that the use of these factor weights in reaching operational determinations were not solely delineated into pass / fail binary decisions (i.e. – whether or not a hedge fund manager was investable from an operational risk perspective), but were also utilized to determine level of conviction in the asset allocation and portfolio construction process.
These operational risk factors are in part based on the amount of perceived risk associated with each factor. For example, within the “Legal/compliance” operational factor area, lax compliance oversight could increase the likelihood of a regulatory violation, which in-turn could cause a hedge fund manager to be shut down. As a result of these potentially serious consequences the “Legal/compliance” operational risk factor area in certain cases may be perceived by fund of hedge funds to carry more risk than say lack of a business continuity and disaster recovery plan. Other factors influencing perceptions as to the appropriate weight a particular operational risk factor should carry include historical incidences of fraud related to certain factors. For example, in the wake of the Bayou fraud an increased weight was placed by many hedge fund allocation organizations on evaluating the independence of a hedge fund’s auditor.
As such, the qualitative data suggests that there was a shifting aggregative tendency among fund of hedge fund organizations to allocate operational due diligence resources towards increasing the comprehensiveness and scope of due diligence efforts within a particular operational risk factor over time.
An example of how this occurs in practice would be a fund of hedge fund organization who merely reviewed the name of the auditor on the audited financial statements of a hedge fund prior to the Bayou fraud. If it was a large organization with which they had prior familiarity (i.e. – a Big Four accounting firm) then they would already have a pre-existing comfort level with this auditor and may perform little to none additional auditor due diligence. Post-Bayou however, this same fund of hedge funds may now, regardless of the service provider’s reputation, attempt to confirm the hedge fund’s relationship with this auditor. Additionally this same fund of hedge funds may take their operational due diligence a step further and incorporate a review of the quality and independence of the auditor into their standard due diligence process.
Data suggests that this aggregative resource allocation trend within individual operational risk factors, has resulted in less operational due diligence resources being dedicated to other different factors which were not the reasons for a historical hedge fund failures. To counter this trend, particularly in the post-Madoff environment, many fund of hedge funds have increased the amount of resources dedicated to operational due diligence via a number of different approaches including reassigning internal resources to operational due diligence, working with third-party operational due diligence consultants via hybrid approaches and hiring additional operational due diligence personnel. Consequently, due to the perceived market importance of the weight of different operational factors as well as a trend of increasing operational sub-factor scope, within each umbrella factor, the total comprehensiveness of a fund of hedge funds operational due diligence process cannot be evaluated in absolute terms solely based on the number of factors analyzed without taking into account the relative scope of sub-factor analysis.
The Madoff Effect
The final stage of analysis was to examine the effect of fraud on hedge fund operational due diligence trends. In particular, what effect would a large-scale fraud, such as Madoff, have on the type of operational due diligence factors analyzed? Three prominent operational red flags which were inherent in the Madoff fraud were studied. These factors are a portion of the core factors outlined in Exhibit 1. This analysis incorporated data from time periods both prior to the Madoff scandal breaking on December 11, 2008 and the post-Madoff period. This limited example indicates a marked increase among fund of hedge funds in reviewing the operational risk factors which were prevalent in the Madoff fraud during the post-Madoff time-period. Particularly, the data indicated an increase in the category of “ Cash controls and management ” an increase of 2.02% from the pre-Madoff to the post-Madoff era. This is followed by similar increases of 1.52% for “ Quality and length of relationship with service providers ” and “ Transparency and reporting ” at 1.63%. These results are outlined in Exhibit 4.
Exhibit 4: Operational factor analysis shifts pre-Madoff and post-Madoff
| Factor Categories | Pre-Madoff Factor Analysis a) | Post-Madoff Factor Analysis b) |
| Cash controls and management | 2.78% | 4.80% |
| Quality and length of relationship with service providers | 4.96% | 6.48% |
| Transparency and reporting | 6.38% | 8.01% |
a) Reflects historical percentages of operational risk factors among the sample set prior to December 11, 2008
b) Percentages are equal to those listed in Exhibit 3 and reflect the period after December 11, 2008
Conclusion
Among fund of hedge funds, there is a wide disparity in both the number and type of operational risk factors analyzed during the underlying hedge fund due diligence process. As compared to those fund of hedge funds which blend the investment due diligence functions with the operational due diligence functions via a shared operational risk framework, fund of hedge funds which dedicate some portion of due diligence resources towards operational due diligence, on average review an increased number of operational risk factors. In regards to the types of operational risk factors analyzed the legal and compliance related factors were the most frequently reviewed operational risk factors, as compared to factors such as board oversight and information technology, which were among the least popular categories. Additionally, the data suggests that hedge fund frauds such as Bayou and Madoff have an effect on influencing the operational risk factors analyzed by fund of hedge funds.
References
About Corgentum
Corgentum is a provider of comprehensive hedge fund operational due diligence services to the hedge fund investors. Corgentum partners with hedge fund investors to improve upon the efficiency and effectiveness of the operational due diligence process. Our proprietary due diligence methodologies and innovative original research allows our clients to continually manage and mitigate their operational risk exposures. The firm’s unbiased and time-tested approach creates a substantial competitive advantage which enhances the risk management functionality of our clients. Corgentum was founded by Jason Scharfman one leading experts in the field of hedge fund operational due diligence and the author of Hedge Fund Operational Due Diligence: Understanding the Risks (John Wiley & Sons 2008). More information is available at www.Corgentum.com.
For More Information
Hedge Fund Operational Due Diligence : The Madoff Effect - An analysis of operational due diligence trends
Corgentum Consulting (235Kb PDF)