Legal Challenge to a Best-Value Procurement System
Publication: Leadership and Management in Engineering
Volume 6, Issue 1
Abstract
The U.S. highway industry has witnessed increased use of design/build project delivery and best-value procurement in the last decade. The industry is experiencing the growing pains of venturing into territory that was previously left to other sectors of design and construction. Inexperience with the system and the processes behind these systems by all parties, state highway agencies, designers, and contractors has occasionally led to protest filed with the court system. The precedents set by the decisions of the court will have a lasting effect on the industry as the exploration of delivery and procurement systems continue. One state highway agency that utilized a two-step best-value procurement process received a protest from one of the firms that was not successful in passing the qualifications phase of the procurement. The court found evidence to uphold the decision of the state highway agency through the transparent and documented evaluation process established by the state highway agency. Continued exploration of new and different technologies and ideas for procurement and delivery of highway projects will result in legal challenges to the system as all parties involved work to gain an understanding of the processes. This paper is an examination of one state highway agency’s experience with an alternative to the highway industry’s typical low-bid procurement system.
Design/bid/build is the traditional method of project delivery for highway projects in the United States. This system is composed of three distinct phases with the project plans being 100 percent designed prior to advertising the project for bids. The low-bid method of procurement has become the norm for design/bid/build projects. The highway method of procurement has become the norm for design/bid/build projects. The highway industry is now exploring alternative project delivery methodologies including design/build. The design/build delivery method allows the owner to award the project with little design complete to an entity that is responsible for completing the design and carrying out construction of the project. Design/build projects are often procured either through low bid or through a best-value structure. Best-value procurement considers not only the cost proposal but qualifications and technical criteria as well. The type of qualifications and technical information requested is dependent on the owner’s needs and how these attributes will be included in the selection process (Beard et al. 2001; Gransberg and Senadheera 1999; Molenaar and Gransberg 2001; Molenaar et al. 1999; Tenah 2001).
The Minnesota Department of Transportation (Mn/DOT) chose to use the design/build project delivery system in conjunction with a two-step best-value procurement method on a project that required the addition of a lane in each direction along eight miles of a well-traveled arterial highway. Mn/DOT completed 30 percent of the design and required the wining proposer to complete the remainder of the design and construct the project. Mn/DOT estimated the project to cost $110 million. The first step of the best-value procurement method, established by Mn/DOT, required Mn/DOT to release a request for qualifications (RFQ). Interested parties were required to submit a statement of qualifications (SOQ). The submittals were evaluated and based on this evaluation all or a limited number (three to five) of the respondents were sent a request for proposals (RFP) in the second step. In response to the RFP the proposers were then required to submit technical and price proposals for evaluation. The evaluations from the first step were not to be carried over to the evaluation process in the second step.
The RFQ included information regarding:The evaluation criteria for the SOQ totaled to one hundred possible points, as detailed in Table 1.Table 1. Evaluation Criteria Established for Mn/DOT SOQ
1.
Minimum qualifications and experience of design/builders;
2.
Scope of work and schedule;
3.
Documents defining the project requirements;
4.
Form of contract;
5.
Weighted selection criteria for the SOQ;
6.
Description of the RFP;
7.
Maximum time allowed for design and construction; and
8.
Cost estimate.
| Evaluation criteria | Possible points |
|---|---|
| Legal and financial qualifications | Pass/fail |
| Organization and experience | 15 |
| Key personnel | 30 |
| Project understanding | 10 |
| Project approach | 25 |
| Project management | 20 |
| Total | 100 |
The RFQ stated: “The selection team shall evaluate the design/build qualifications of responding firms and shall compile a short list of no more than five most highly qualified firms in accordance with qualifications criteria described in the request for qualifications (RFQ).” Each evaluation criteria included the main points of assessment. Under the organization and experience category listed in Table 1, consideration points included:This section specifically directed the respondents to “describe at least one but a maximum of four design/build projects that the Submitter and each Major participant has managed, designed and/or constructed.” The RFQ also stated that respondents should “Highlight experience relevant to the Project that the Submitter/Major Participants have gained in the last ten years. Cite projects with levels of scope comparable to that anticipated for the Project.” Under the “key personnel” category the RFQ requested the SOQ include the name and resume of eighteen required positions. The evaluation criteria for this section included team members with experience and qualifications that cover project scope and key management/staff experience, capabilities, and functions on similar projects.
•
Effective project management authority and structure
•
Design and construction management structure
•
Effective utilization of personnel
•
Owner/client references
•
Experience on projects of similar scope and complexity
•
Experience with timely completion of comparable projects
•
Experience with on-budget completion of comparable projects
•
Experience with integrating design and construction activities
•
Experience of design/build team members working together
After release of the RFP, Mn/DOT released an addendum that replaced the description statement presented earlier with the following: “Describe at least one but a maximum of four design/build or major highway projects that the Submitter and each Major Participant has managed, designed, and/or constructed.” This second statement request thus eliminated the need for any design/build experience.
Five SOQs were received by Mn/DOT. Upon receipt Mn/DOT convened the evaluation committee, composed of six members. Each member was trained on how to score the SOQs. An evaluation plan had previously been composed with the stated purpose of providing the methodology and procedures for the evaluation of the SOQs. The plan called for each SOQ to be evaluated by each committee member. Each committee member evaluated each section of the SOQ based on the degree of satisfaction of the requirements set forth in the RFQ. Each committee member was also asked to identify potential strengths and weakness of the SOQ. Each committee member was asked to assign an adjectival rating of excellent, very good, good, fair, or poor to each category of each SOQ. The evaluation plan included how each member was to assign the adjectival rating based on the number and level of strengths and weakness of the submittal. Based on the adjectival rating a percent score was determined; this percent score was then multiplied by the maximum score for that category to determine the final score for that category. Each category was then combined for the committee members’ total score for each SOQ. The averages of the combined scores for each SOQ were then determined, as shown in Table 2.Table 2. Results of SOQ Evaluation
| Submitter | Average score |
|---|---|
| A | 85.7 |
| B | 83.0 |
| C | 82.7 |
| D | 71.9 |
| E | 69.4 |
The evaluation team determined that the top three SOQ respondents would receive an RFP due to the large break between the scores (over ten points) between the top three and the bottom two submitters. Submitter D, with the forth-highest score, submitted a protest to the short list outcome. The protest was filed for the following stated reasons:
1.
Mn/DOT violated the state design/build statue by requiring that the evaluation would rely on and emphasize previous design/build experience, which would restrict competition.
2.
Mn/DOT engaged in unpublicized rule making.
3.
Judgment regarding who was short-listed was arbitrary and capricious fact-finding, and the conclusions were not substantiated by the evidence.
The courts found that the plaintiff, Submitter D, did not prove that Mn/DOT created a restricted competition situation by requiring design/build experience. The RFQ did state that design/build experience would be considered, but not required. Mn/DOT also never directly or indirectly instructed the evaluation committee to weight design/build experience differently than similar project experience. The court did note that individual evaluators might have come to this conclusion based on their own judgment.
Given the limited experience with design/build by Mn/DOT and the unique features of the project the court found that establishing rules for evaluation on a case-by-case basis is reasonable. Also, the rules established on this project do not formally establish policies for any future Mn/DOT design/build projects.
Finally, the court system found that Mn/DOT did not act arbitrarily and upheld the Mn/DOT short list decision. The evaluation process was documented with expressed standards and while the process does not include the safeguards of low-bid competitive bidding, the court could not conclude that the process did not include reasoned decision making. The court could also find no reason to challenge the members of the evaluation committee as not being qualified and experienced enough to make the short list decision.
The court concluded that no one submitter was favored over another, and while the evaluation technique may not be perfect it is reasonable. The court ruled in favor of Mn/DOT and left the short list decision intact with no restitution for the protesting submitter.
Commentary I
Douglas D. Gransberg, M.ASCE
This case sets a very important precedent for not only design/build projects but also all projects procured on a best-value basis. No matter how careful an owner is in articulating the details of their best-value award evaluation system, there will always be the possibility that one or more of the potential proposers will be dissatisfied with the outcome and file a protest. The courts in this case upheld the right of the owner to determine what makes one offer a better value than another offer. I believe that this case was won for three major reasons:
1.
The SOQ evaluation criteria were transparent to all offerors.
2.
The owner followed its evaluation plan as published.
3.
The owner could defend its decision logically.
Transparency in evaluation plans is a key element of best-value success. Parvin, a lawyer who defends construction contractors, wrote an insightful article regarding design/build evaluation planning for highway construction projects (Parvin 2000). The article discussed the legal view of the need for fair and open evaluation processes in the transportation industry. In it, he highlighted the following two points that are of interest in this case:He argued that without a transparent evaluation plan, the owner would find itself constantly defending award protests. In this case, Mn/DOT clearly articulated the definition of success. It helped its case by stating that it would narrow down the field to a short list of the “most highly qualified firms in accordance with qualifications criteria described in the request for qualifications (RFQ).” The requirements for making the short list were clear, and each potential competitor could compare itself to the competition and make an informed decision as to its ultimate competitiveness in the known field of players. This transparency serves to reduce the element of subjectivity that is inherent to best-value award evaluation systems by spelling it out rather than hiding it.
1.
“Clearly state the evaluation criteria and weight given for each item and ensure that the evaluation team uses them.”
2.
“Leave no doubt about the honesty and integrity of the public agency’s evaluation team.”
Secondly, as the requirements for making the short-list were clear, it was easy for the courts to find that Mn/DOT had followed its own evaluation plan. This speaks to the second Parvin quote cited above. Once an owner publishes its evaluation plan, it loses all flexibility in applying it to the proposals that are received. If it wants to be able to defend against a protest, it must follow its own rules. If it does, the second Parvin quote regarding leaving no doubt about the honesty and integrity of the evaluation team is satisfied.
Finally, the use of the terms “most highly qualified” gave Mn/DOT great latitude about determining the final size of the short list. The resultant scores showed that three competitors fell within a range of three points of each other, while Submitter D was ten points below the third-highest score. This created a solid argument that the three competitors on the short list were indeed the most highly qualified. In fact, Mn/DOT was unintentionally doing Submitter D a favor by not being conservative in forming the short list with four firms. The cost of preparing a technical and price proposal for the $110 million project would probably be in the range of $300,000 to $500,000. If Mn/DOT had arbitrarily set a minimum point score of seventy to be considered “qualified” (i.e., minimally qualified), then three instead of two firms would have had to invest that significant amount of money in a losing effort. Thus, the logic of short-listing the only “most highly qualified” instead of all the minimally qualified is compelling in an economic sense and in the long run is much more fair to the design/build industry.
Thus, several lessons can be learned from this case. First, the owner must publish transparent evaluation criteria and its best-value award algorithm. This puts all the competitors on an even footing and makes the defense against a possible protest stronger. Secondly, once published, the owner must follow its evaluation plan to the letter, collecting documentation along the way to prove that the decisions made for the project flow directly from the published evaluation plan and its attendant criteria. Finally, the plan must be logical and the decisions that flow out of it must also be based on defensible logic. In application, the use of the term “most highly qualified” probably is advisable in any RFQ evaluation plan as it gives the owner the greatest amount of latitude to develop a short list while being fair to those firms that could not possibly win the project on a basis of qualifications alone.
Commentary II
Keith R. Molenaar, A.M.ASCE
This case illustrates why two-phase best-value procurement is necessary for design/build delivery and also illustrates the challenges that are inherent in the system. Since design/build highway procurements often require substantial design effort, the proposals are costly to prepare. Public sector agencies must keep industry proposal costs to a minimum if they wish to make design/build a sustainable alternative delivery method in the transportation sector. Two-phase procurement provides a simple and low-cost method to minimize industry proposal costs by creating a shortlist of only those proposers who have the highest potential for success. In issuing an RFQ and short listing to only three proposers, Mn/DOT minimized proposal costs to the industry. Two-phase design/build procurement has been proven successful by many other government agencies and is endorsed by the Federal Acquisitions Regulations (FAR 2005; Molenaar et al. 1999).
A primary challenge in using two-phase best-value procurement is that the use of subjective evaluation criteria is unavoidable. The use of subjective evaluation criteria is commonplace in consultant selection, but is foreign to most highway contractors. The subjective criteria that Mn/DOT chose to use (i.e., design and construction management structure, effective utilization of personnel, experience on projects of similar size and complexity, etc.) are very similar to the criteria that engineers encounter when preparing project proposals. Since design/build projects contain substantial design work, these criteria must also be applied to design/builder selection in addition to the actual cost of the project.
The judge commented on the subjective nature of the management evaluation criteria in his ruling on the arbitrary and capricious allegation. He stated that the members of the technical review committee (TRC) were qualified and any errors in scoring “would not have changed the outcome.” The judge found that “it was ultimately the responsibility of the submitters to provide the necessary information and to convince the committee that it was the most qualified Submitter for the [design/build] Project,” and while the TRC “drew conclusions that, while not perfect, were reasonable, … their decision does not reflect an error of law, arbitrary and capricious fact-finding, or conclusions unsubstantiated by the evidence, … [and they did] not exceed [their] statutory authority or abuse [their] discretion.” However, the judge did state that the evaluation process could be improved. “It would be impossible to use people in the process and filter out subjective evaluations … the fact that the process could be improved does not make the process used in this case arbitrary or capricious.”
If highway agencies wish to use design/build project delivery, there will need to be some level of subjectivity in the procurement process. Design/build contracts are contracts for both services and real assets. The assessment of services to be performed is by nature a subjective evaluation. It is up to the highway agency to be sure that the evaluation is reasonable and directly relates to the published evaluation criteria. Owners must use experienced design and construction professionals to evaluate proposals and then provide a documented evaluation plan for them to follow. The judgment of these professionals is an acceptable method in which to choose a design/build team given that they follow the evaluation plan and document the reasoning behind their judgment.
Finally, if the owner is going to evaluate a specific factor, such as past design/build experience, this should be stated explicitly in the published evaluation criteria. Although the judge did find that “experience on projects of similar scope and complexity” did equate to design/build experience, the owner and the industry would have been better served if the RFP directly stated design/build experience in the description of the evaluation criteria. Owners are best served by making evaluation criteria as transparent as possible at the time of evaluation and not deviating from these criteria during evaluation.
Commentary III
Joseph R. Gladke, P.E.
This case demonstrates the importance that owners must place on having a procurement process that is well documented, in compliance with state statutes, and contains clearly stated evaluation criteria. Design/build projects are not typical construction projects. There are specific reasons why projects are chosen as design/build projects, and specific goals established for each project. These project goals, along with weighted evaluation criteria, are stated in the RFQ to help potential design/build teams understand what the department is trying to achieve with the project. Through this understanding, contractors and designers form teams comprised of individuals with the experience to meet the goals of the project. The teams submit SOQs detailing their qualifications, organization, and project approach. Following the evaluation criteria, the department must determine which teams, typically three to five, are most likely to meet the project goals and deliver a successful project. It is in the best interest of the public to have a well-qualified design/build team to help insure the project goals are met. Through the short-listing process the most highly qualified teams are selected to prepare project price and technical proposals. Using a traditional design/bid/build process the experience and past performance of the contractor is not considered. The only requirement that must be met to bid on the project is having the necessary bonding capacity, and thus the likelihood of the contractor delivering a successful project that meets the goals is unknown.
One of the claims in the lawsuit was that Mn/DOT violated the state design/build statue by requiring that the evaluation would rely on and emphasize previous design/build experience, which would restrict competition. Mn/DOT did not violate the design/build statute. Mn/DOT followed the statute that outlines the requirements of the RFQ, including “requirements for construction experience, design experience, financial, personnel, and equipment resources available from potential design-builders for the project and experience in other design-build transportation projects or similar projects, provided that these requirements may not unduly restrict competition” (Minnesota Revisor of Statutes 2001,2004).
Short-listing three to five teams does not unduly restrict competition, and it increases the likelihood of having a design/build team that can deliver a successful project that meets the project goals. Mn/DOT has no interest in restricting competition. Mn/DOT has an interest in having more highly qualified teams. In an effort to obtain more highly qualified teams, Mn/DOT offers debriefing meetings with unsuccessful teams to help them understand what improvements they could have made in their SOQs or technical proposals to obtain a higher score. Mn/DOT also makes SOQs, technical proposals, and price proposals from past projects available in its library for review.
Finally, design/build procurement is not fully understood or accepted by some contractors. Some of them have been very successful in obtaining projects under the design/bid/build procurement approach, yet they may need to modify their business processes, procedures, personnel, and bidding approach if they are to be successful on design/build projects. Owners are well served by helping new design/build teams understand improvements that they can make on past SOQs and technical proposals in an effort to have more teams that better understand the design/build procurement process and can improve on their scores for future projects. Design/build is not about limiting competition; rather, it is about having well-qualified teams that can meet project goals and ensure successful projects for a reasonable price (not necessarily the lowest initial bid).
References
Beard, J. L., Loulakis, M. C., and Wundram, E. C. (2001). Design build: Planning through development, McGraw-Hill, New York.
Federal Acquisitions Regulations (FAR). (2005). “Subpart 36.3—Two-phase design-build selection procedures.” Federal Acquisitions Regulations, ⟨http://www.acqnet.gov/far/⟩ (July 5, 2005).
Gransberg, D. D., and Senadheera, S. P. (1999). “Design-build contract award methods for transportation projects.” J. Transp. Eng., 125(6), 565–567.
Minnesota Revisor of Statutes (MRS). (2001). Minnesota session laws 2001, 1st special session, chapter 8, article 3, section 5, MRS, Saint Paul, Minn.
Minnesota Revisor of Statutes (MRS). (2004). Minnesota Statutes 2004, section 161.3420, subdivision 3, MRS, Saint Paul, Minn.
Molenaar, K. R., and Gransberg, D. D. (2001). “Design-builder selection for small highway projects.” Eng. Manage. J., 17(4), 214–223.
Molenaar, K. R., Songer, A. D., and Barash, M. (1999). “Public sector design/build evaluation and performance.” Eng. Manage. J., 15(2), 54–62.
Parvin, C. (2000). “Design-build: Evaluation and award,” Roads and Bridges, 38(12), 12.
Tenah, K. A. (2001). “Project delivery systems for construction: An overview.” Cost Engineering, AACE International, 43(1), 30–36.
Biographies
Jennifer Shane is a PhD candidate in the Department of Civil, Environmental, and Architectural Engineering at the University of Colorado, Boulder, and can be reached via e-mail at [email protected]. Douglas D. Gransberg is associate professor in the Construction Science Division at the University of Oklahoma. Keith R. Molenaar is assistant professor in the Department of Civil, Environmental, and Architectural Engineering at the University of Colorado, Boulder, and can be reached via e-mail at [email protected]. Joseph R. Gladke is design/build program director at the Minnesota Department of Transportation, and can be reached via e-mail at [email protected].
Information & Authors
Information
Published In
Leadership and Management in Engineering
Volume 6 • Issue 1 • January 2006
Pages: 20 - 25
Copyright
© 2006 ASCE.
History
Published online: Jan 1, 2006
Published in print: Jan 2006
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.