Comprehensive Debriefing on DBITS Solicitation Process and Evaluation

Welcome everyone. We're going to get started in  just a second. We're just letting everyone log on. Again, if you're just joining us, we're  going to get started in just a minute.

again. If you're just joining us, we're  going to get started in just a minute. If you're just joining us,   we're going to get started in just a  minute. We're letting everyone log on.

All right, welcome everyone to the debriefing  conference for deliverables based information   technology services. Before we get started,  we're going to go over a few webinar notes. I   would like to remind everyone that you can switch  between your computer audio and your phone call.  

And if everybody can please raise their hand  to confirm that they can hear me. Perfect.   Thank you all so much. I would like to remind  everyone that this is a debriefing conference,   so the attendees will be muted  and no questions will be allowed.

And I will now turn it over to Vania. Hello everybody. Welcome. And I want to welcome  everyone here. I want to introduce my director   uh Stephanie Harrison. She is the person  that pretty much supports us and help us  

get all of this work completed. I also  wanted to introduce Julia. She is my   um she's the procurement lead that is working  with me. I'm the primary. She is my backup.   So throughout this process, we have worked  together for a successful solicitation process,  

which we are at this point. Now, I like for you to  know that this wasn't a one person project. It has   taken a village and I'm going to go through the  process so that everyone can understand exactly   how DIR have reviewed and evaluated the responses  for the DBITS solicitation. Next slide please. So  

here's our agenda. I'm going to talk to you about  the purpose of the debriefing. I don't want anyone   to be confused. This is not an individual um  debriefing for any vendor. It is an overall review   of what has been happened. Then I'm going to talk  about the objective. We're going to go over the  

detail of the administrator review. What is a  responsive review? What's a responsible review?   We're going to talk about how the evaluations  were done. Then we're going to see the areas   that the overall improvement that we have found  throughout the proposals. Nothing in particular  

to one vendor. Also, we're going to go over what  the respondents should expect to happen next,   and we'll do some closing statements to go over  what we expect from you guys as we move along.   Next slide. So, this journey began October 2024.  For those vendors that were with us, we started  

doing a vendor forum. We did vendor trainings.  We did a customer forum. We did pre-solicitation   conferences, two of them for the transparency  and communication of this process. In addition,   the procurement staff have been available and  have answered over 624 question throughout the  

solicitation Q&A period and we continue to answer  question even after the Q&A period was over. And I'm saying all of this to you to tell you to  demonstrate that DIR has gone above and beyond to   provide all the needed information, all the  needed tools for each vendor to prepare and  

complete a successful response. So the purpose  of this debrief is to provide respondents   with information on what DIR has learned after  conducting a detailed administrator review of 357 responses to minimize the number of inquiries on  results of this solicitation review and evaluation  

and to provide response respondents with the Next  steps on the schedule of this solicitation. Most   important on the purpose is that the next  phases of this solicitation is the critical   negotiation process. And it is important that  the procurement leads are not bogged down with  

answering questions and emails. that is going  to be taken away from us providing you guys   with a fast negotiation timeline. So I'm hoping  to answer all your questions today. Next slide. So, the objective of this is we're going to  ask each vendor to please pull your response  

packages. As we go through this presentation,  I want you to review what you submitted. I want   you to look at your response packages. Okay?  And when I say this document is required,   this document is required throughout the process.  As a responsive vendor, you will be required to  

have the documents I am going to discuss.  These document information need to meet the   requirements to be considered responsible. So  we are going to discuss what that is and then   I want every vendor to understand the DIR review  process how detailed and how labor and intensive  

it has been but we took the time to do it very  carefully and do it right and to for the vendors   to understand the next step on the process and  what they're going to receive from us. Next slide. So what's a responsive review? Next slide. So  based on the procurement guide from the CPA,  

a responsive proposal is one that fully adheres  to all the documents requirements outlined in a   request for offer. That means that every  document that we requested you provided   and each one of documents are completed are  signed according to the RFO instructions. Okay.  

And any additional required documentation are  provided and clearly labeled that the company   establishment documentation was provided to  meet the requirements. Next slide. So with your   proposal in front of you, each one of you can look  at your list of documents that was uploaded to the  

VIS portal. Please do not look at the documents in  your computer that you think you submitted or that   you think you should have submitted. Look at your  VIS portal uploaded documentation because that's   all I had to go by. So the very first document you  needed to provide me is your exhibit A with the  

attachments and signature. If you do not see that  document, you've been disqualified. Okay. Second,   exhibit A1 exceptions. This document needed  to be submitted with an N/A as no exceptions   or with your exceptions. If a vendor did  not provide an exhibit A1 with exceptions,  

DIIR will not negotiate additional exceptions  and negotiations. We are going to review only   what was provided during the response time. So,  make sure that you understand that because if   you thought you uploaded an exhibit A exemption  exceptions and it's blank because you uploaded the  

wrong one or if you refer on your appendix on the  exhibit A says response document is attached but   that document was never attached. Just because  you wrote it on the exhibit A does not mean   I'm going to accept your exceptions. So let's  take that right off the table so that everybody  

understand that requirement. Exhibit B, your hub  uh is the respondent history and experience. The Uh, looks like we're having some technical  difficulties. Give us just a minute. Hear   me? My computer just gave me a notice that I  was having problems with my microphone. So,  

I'm checking. We can we can hear you now. Uh,  Patti, can you check to make sure everybody can   hear me? Um, if you have good audio connection,  raise your hands. So exhibit C contract marketing   and support plan. Can you go can you go that  attachment was required. It was another most  

important evaluation piece of documentation.  You needed to provide two attachments with this   exhibit C. Not everybody did that. So I want to  make sure you understand if those documents are   not there we could not evaluate you. Okay, that  was an automatic zero on that section. So that  

you know number five is your hub subcontracting  plan with all of your documentation. So method A,   method B, whatever method you chose, you should  have set up all the backup. Okay? And we'll talk   more about that later. I guess I need to go back  to exhibit B. Apparently, the microphone wasn't  

working at the time. So, exhibit B, respondent  history and experience, the most important   document on your solicitation package. 60% of  your score. If you did not submit this document,   you will not be evaluated under that area.  That is 60% of your score. So please note in  

your document set if you did not submit an exhibit  B. It is a very critical document and there were   vendors that did not. So that you know there was  also vendors that did submit exhibit C and I'm   sure it could have been an error. It could have  been whatever reason but you didn't. So there's  

no way to go back. Okay. Let's go to exhibit F.  Release of liability. This is a document that if   you provide it, it can be useful. If you didn't,  it is okay. Okay. Depending on your references,   if we needed to speak to them, if we didn't have a  release of liability from you that we could speak  

to your reference, we could not do that. So, that  could uh also adjust your score. So, be aware of   that. But it's not document that you didn't have  to do it and you are okay if you didn't. But it   does affect how we treat your references so that  you know. Exhibit G. This is a critical document.  

Everybody take notes and make sure you see it  in your package. But here's the deal with the   exhibit G. your spreadsheet PDAA accessibility had  a very important fail or pass portion and we'll   talk more about that but take a look at it some  of you did not submit this form exhibit H your  

accessibility VADSIR this was a requirement for  only vendors proposing category one development   There was 160 vendors that proposed category  one. So for those VASERS will be reviewed. Number nine, the most important document of your  whole package. This was part of addendum one.  

It was a requirement. Every company must submit  establishment document. We gave you some samples   of what kind of documents, incorporation  documents, articles. There was different   types of documents, but the the key to this is to  establishment date proof that could be verified.  

That was how we provided you that information.  And vendors that did not propose this document   in their solicitation were disqualified.  This was a must requirement. Next slide. So now we're going to talk about what is  considered a responsible review. So most  

vendors see res only look at the section that  talks about responsive. Sometimes uh when a   solicitation document talks about responsive  um responses and proposals they talk about both   responsive and responsible but most of the time we  don't different the two. The contract management  

guide different as follow. Responsive means you  provided all the documents. Responsible means you   meet the requirements with those documents. Okay?  So if you just provide all those documents but you   don't meet the requirements, you're not going to  pass it. Okay? So what are we looking at first?  

You have a immediately administrator review pass  or fail. You responsive and you provided all the   documents. You pass. So now we're going to look at  responsible. Do you provide your hub form? Well,   did you provide your PDAA? Did you have your  PDAA completed? Did you have your financial  

um Dun's number in your and was it correct and did  it pass the scoring requirement? Did your company   establishment document that you provided does it  have the five years requirement if you provided   category one or three or more years if you had two  three and four that's being responsible you meet  

those requirements and then there was the scope  review and we're going to talk more about that   in the next slides next slide. So document that  gave you the instructions for these documents   the way the RFO reads section 3.8 respondents  will follow the instructions set forth in the  

following exhibits attachments to this RFO. So you  were giving every single instructions for every   single document to how to fill it out, how to  upload it. You were giving detailed instructions   what was required, how to respond. Respondents  response must be consistent with the structure,  

sequency and terminology used in this RFO. Those  are very important instruction that sometimes   people just go over it and don't understand it  and then your response reflects that. Okay. So   must be aware of these requirements. So on the  exhibit A what was the requirement? Next slide.

So exhibit A section 4.1 the evaluation  of response amendment through addendum   number four. Okay. We talked about  we replaced a piece of this, right?   we added additional documents and in this  addendum we answer questions as well.  

So all of these things we talked about we talked  about evaluation response being responsible the   scope of the RFO the required documents and most  important the pass and fail it's right here very   clear talks about the PDAA must be completed  zero PDAA scores were not going to pass no  

PDAA form was not going to Yes. So over here you  got the instructions and everywhere else during   this process and all the communications provided  to you, you receive more directions. Next slide.   So exhibit A must have been submitted, must have  been signed. So if you're looking at your response  

package, did you fulfill those requirements? DIR  provided this information on several parts of the   RFO documentation. The response shall complete  and sign the executive form. Any exhibit A that   is not signed, your response will not qualify  to move forward. We also require that you use  

our file convention names and we gave you this  table in several different areas of the RFO very   consistently to remind you of these documented  and the signatures requirements. Next slide. Then you have your B and C and A1.

We told you this on exhibit A item 11.  Exhibit B throughout the RFO document.   Exhibit C attachment one and two. All respondents  must provide detailed response to each question   detailing the respondents history and experience  in providing the services proposed based on the  

categories that you are proposing. Okay. very  important information that needed to follow. So   not answering a question under the question item  and providing information throughout the form   itself without following the instructions  is not considerate answering a question.  

You must understand how the evaluation of  an exhibit B and C works so that when we   go through your evaluation, you can see  where you fell short. You cannot just   provide us a document where you talk  about things all throughout 15 pages,  

but you're not answering any of the questions.  So, you know exactly how that worked. Next slide.   The PDAA qualification this qualification  was clearly specified on 4.2 item D and also   on conference slide from April 9th. Okay. And we  specifically said failure to complete and submit  

your PDAA, it is an automatic disqualification.  Anyone that submitted a PDA form with a zero score   or a blank form also failed the PDAA. Okay. So  basically in this class, six of the vendors fail   this area. Six of you were DQed right here  due to these two requirements. Next slide.

providing a uh no go back one please. Hold on. Okay. Yes. Go to the next slide.

Okay. There we go. So, hub plan disqualification  section 3.4.1 of the RFO. What are the   requirements? You were disqualified.  One, no hub plans submitted. Two,  

you completed the wrong method. If based  on your company, you either had to do A   or B. If you completed the wrong one, you did  not pass this. you did not provide the backup   documentation for the appropriate method. So  on the hub solicitation, so that you know,  

251 vendors out of 357 companies were in what  we call clarification status for the period of   the review and evaluation, which we worked  with each individual vendor to resolve your   issues so that you would move to evaluations.  Only three vendors were DQed at this stage.

Next slide. Company establishment documentation  disqualification. So we explained to you on   addendum one. We put it in several of the  slides on our pre-proposal conference.   We discussed it and in every question on all  the addendums we explained that this document  

was a requirement and it was not a maybe it was  not a it was a requirement and it's very clear   when it says must provide this document not may  not should must. Next slide. So, addendum one,   we talked about it again, they will provide as  part of their response documentation. Next slide.  

We talked about it again. So, basically, this  was a very important piece of document that was   completely different from most of the RF in the  past. So, a lot of people that were accustomed to   DIR way of doing things or thought they knew and  didn't read the requirements did not provide what  

was required for this document and 27 vendors  were disqualified for not providing a company   establishment document with your response. So  if you look at your response right now like I   asked you earlier to review your bidstamp  you're VIS portal you're at your and you  

look at all the documents that you provided  us if you don't see a company establishment   document of whatever kind it was if you did  not pro provide it you were disqualified and   did not move to negotiations to evaluations very  simple and as you can see based on the language  

And on all the information provided, there's  no room for misunderstanding here. We either   had your document and were able to verify that  you had your five years or your 3 years or you   did not have a document and we did not pass  you. Very simple, straightforward. Next slide.

Next slide. All right. Next, financial review.  So the requirement was that at the time the DUN's   report was run for the financial review of the  solicitation your SER score was to be nine or   above. I mean nine or below. Sorry that should  have said below. I will correct that. So if you  

had a nine or above you fail. And there was only  two vendors that failed this part of and there   were 76 vendors that were under clarification  that did not have a three-way match, which was   very critical to the review of your financial  review. In order to pass financial review, you  

were to have a score nine or below. If you had a  nine or above, you fail. You were supposed to have   a three-way match. And I remember I told you guys  a four-way match is my preference, but three-way   will do it. Some of you did not have a three-way  match. And we work together and we work with these  

76 vendors to make sure that everybody passed.  Okay? And we worked very close with everyone and   we made it through it and only two vendors fell  financial because of the SER score at nine. Okay.   Next slide. So what was the information that  must have match? Your legal name, your corporate  

address, your fan number, and your Dun's number.  Those things needed to match between your response   document exhibit A, your VIS portal, and your SOS  registration all along with the Dun's number. So a   lot of you I'm very very happy that throughout the  process of what we worked together during the the  

preparation of this solicitation we only had 76  issues and that is a record for 357 responses and   we got through all of them and it was pretty quick  comparing to the numbers. Next slide. So next   we're going to do an RFO scope review. That's what  we did next. So we work through all this process  

with every each response and we were separating  the ones that make it the ones that don't and we   put them and now it's time for the remainder  vendors that have passed financial hub PDAA   and requirement documentation to review scope.  So there was four categories and how did DIIR  

review the scope? So the number one next slide the  most important question of all the documents you   provided us was exhibit B question number three  and your references. Why? Because DIR question was   very clear using this document and in no more than  three pages per category that you are proposing.  

So if you propose R4, I'm going to read 12 pages,  right? Provide a summary of the respondent's   experience with a focus on projects completed  within the five years. Not older than five,   within five years. If serving as a subcontractor  responded surely clearly indicate the percentage  

of the work for which it was directly responsible.  No marketing materials and no links to website was   allowed. So what was the we're looking for here  guys? We're looking for projects. We were looking   for you to list the project. Give us the project  name. Maybe if you wanted to the customer name,  

but give us the project scope, give us the project  budget, give us the project timeline, and what was   that you did that show us your experience that  meets this RFO scope. Basically, this question   is what the rubber meets the road and it told us  everything we needed to know. Next question. So,  

what did we do? We had three different people,  very qualified individuals that looked at every   single one of your response. Exhibit B, question  three, along with your references to notice and   make sure that you had what it needed to  meet the qualifications of your experience.  

based on what you propose. Okay? So, if you wrote  your company history and in your company history,   you talked about all kinds of stuff, but you  fail to show us how the experience of your   company within a project. If you spend most of  your time giving us information on question one  

and you did not give us what we needed, which  was your actual project experience to show that   you met the qualifications, you did not answer  the question. Okay. And in some cases, we looked   at your references to validate your project  information that you provided on question one.  

We didn't just do this once, guys. We did this  three times. And we had three different people   look at it. And then after that, we had vendors  that did not provide projects that answer question   number one and three with marketing propaganda  information. We had vendors that simply did  

not answer the question at all. Okay? And a lot  of vendors provided us with projects that were   older than five years, which we took a little  bit in consideration, but we couldn't take it   all because the requirement was within the last  five years. Now, there were vendors that provided  

us with five projects that were ongoing past  the five years. So, they started way before,   but they're still active. We considered  those because they're still active, even   though they started a lot longer than 5 years,  but they hadn't finished yet. So, next slide.

So, one of the things we looked for while  reviewing scope is if you had any exclusions   incorporated into your responses. So, if you  had any cloud broker services experience,   it didn't count. If you had professional  consulting services experience, it didn't count.  

What else didn't count? If you had projects  that simply told me that you did software   maintenance for a customer, that didn't count.  It didn't count for the whole experience. Okay,   we explained this many times that DBITS  is a comprehensive services. DIR was not  

going to award software maintenance by  itself or sec cloud security by itself.   We were awarding the category and you can  do anything within those categories but   you had to qualify for the category as a whole.  Okay, very important to understand. Next slide.

So here's the final results of what  we did. Admin initiated review,   responsive and responsible review. 357 vendors,  50 total responses were disqualified. What make   those numbers? Nine administrator review round  one missing documentation. 30 administry review  

round two either didn't have the sufficient  years of experience or didn't provide a company   documentation three two financial review  three hub and six accessibility those are   the numbers of all the disqualifications  okay 50 total did not move to evaluations

next slide So, now we're going to talk about  evaluations. And I think we're pretty good in   time. I don't believe we need a break yet. And  I am hoping to get this done a lot sooner than   two hours so that we don't need to take a break  and we can save some time. So, now we're going  

to talk about evaluations. Their team responsible  for the evaluations dedicated 100% of their time   to complete these evaluations in a period of  eight weeks. Okay, I want you to understand the   volume. There were 357 37 move to evaluations.  We were reviewing 18 questions per vendor.

Almost 10,000 questions were reviewed. Okay.  So, we put a lot of time in a lot of effort   and the evaluating team was an amazing team,  super qualified. We had a technical expert.   We had a contract management expert and we had a  procurement person totally very an expert on the  

scope of this RFO because this person been  with the project since October. So totally   knowledgeable about the requirements. So I'm  very happy that we had a superb evaluating   team. Now what did we evaluate it? So  your exhibit B was 60% of your score.  

So based on the information I've been giving you,  you pretty much know how you did and exhibit C   40% of your score. Right? So what are the things  that we found the most on this document missing?   People just didn't get part two very well.  Everybody got marketing plan. See that you  

all know how to market. But the second part  was a little more difficult and we did find   some good ones but we had some struggles  here and the most important part on this   section. A lot of you did not provide us the  right attachments for exhibit C. Next slide.

So see can you go back one? Okay. Yes. Thank you. So at a high level overview  we consider your information submitted with your   response forms. Any information provided  outside of your response or over the page  

limits were not considered. Okay, we look  at your history and experience. All of the   people that received references, they  were combined with your exhibit B. We   reviewed your experience to add points to your  evaluation. But if you did not have references,  

it did not affect your evaluation  on a negative way. Next slide. So under the evaluations there is a very  point that is very important. We told every   the vendor community more than once that  you did not need to be fully registered  

and active in the state of Texas to receive  a contract. But we did tell you that at this   point intent to negotiate that you would  have to be registered and active in order   to move to get a contract. 14 of the vendors  did not have this requirement met. Therefore,  

they will not move to negotiation even though  they made the competitive break. Okay. Our   competitive break was marked as 75. Why is the  bids we require quality? We require vendors   that are very high level to do the work that this  contract solicitation scope requires. Next slide.

So basically what are the areas of improvement   that I recommend for our vendor  community on the DBITS project. Some respondents did not follow  instructions. Company history  

did not tell a chronological story of the  company progression in growth and experience.   you just a lot of you just did a lot of  advertisement and marketing of your company   and your company history. Some respondents did  not demonstrate the years of experience for  

the category they were pursuing. Some  respondents failed to provide actual   relevant details regarding completed projects in  relation to the categories they were pursuing.   Some respondents did not provide project  references to the categories they were  

pursuing or provided projects older than 5  years. So follow instructions understanding   requirements. Some respondents failed to  answer the questions completely. Next slide. So, appendix C, what areas of improvement on  your customer plan? Some respondents did not  

address all the questions. You left some of  the questions blank. Some respondents offer   emphasis on recruiting talent, but failed to  demonstrate the actual talent and skill they   have on board to fulfill a scope of work  right away. Since this was not an ITSAC,  

we were not looking for anyone to  recruit talent to do the job. Yes,   there would be some subcontracting absolutely on  some of the scope of the work, but we're looking   for companies ready to go when they get a scope of  work. Some respondents failed to answer the actual  

question and provided information not relevant to  the question or scope of work. Some respondents   provide a very generic org chart and failed  to address all the required areas in that org   chart. And most important, very few show the DIIR  team managing the contract within their org chart.

Some respondent did not provide attachments  at all and some respondents did not provide   a sample of an actual SLA and a lot of vendors  missed answering question five which was supposed   to go along with the two attachments. Missed that  completely. Okay, next slide. So now we're going  

to talk about how the competitive break came along  and what we did next. So the high-level overview   of a competitive break is when our responses  were scored. DIR determined the competitive   breaks existed. DIR did not have a predetermined  number of awards. The competitive break were  

determined for each of the four major categories.  The DIR use a set methodology for determine where   the break occur mathematically arranging the  scores in the orders of highest to lowest and   determining the average between each response  scores to find the break. Okay, next slide.

So the categories of category one is application  development. We had several different sets   combination of category one with two and three,  category 1 with three and four. Basically, we took   category 1. So, we had 272 vendors that proposed  category 1. Remember I told you 160 early proposed  

all four categories, right? But out of 100 out of  all of these 272 only 155 made a competitive break and the break is vendors that propose all four 95  and so forth and so on based on the categories.   There are vendors that propose just one category.  Okay. So these are the numbers that came from  

category 1 and the category 1 sets. Next slide. So  category 2 vendors that proposed category 2 with   different combinations. There were 13 that didn't  propose category one that proposed category two. 13 vendors made that intent to negotiate 23 total  and the score was 75.34 where the break occurred.  

Okay, next slide. So category three only or three  and four. Can vendors propose category three?   No category one, no category 2, just three  and four or three by itself. Only three made   the competitive break at 75.33. So the as you  can see the overall break is at 75 or above.  

Right. Next slide. For category four, we only  had one vendor that only proposed category 4,   but he did not make the 75 competitive break.  Therefore, we did not award any vendor that   proposed category 4 only. But there's plenty  of vendors proposing category 4 in different  

combinations all through. And we fulfill our  requirements that have at least six vendors   for our customers to use with any given of their  thresholds of requirements for procurement on our   categories. Next slide. So what is next? We have  done the job. We have worked. We have finished.  

And now we're going to go to the next step.  Okay, next slide. So vendors will be notified hopefully by tomorrow at 5. You will get your  notices if you were DQed, you missed the intent   to negotiate and if you're moving to negotiations.  Those three email sets are going to go out as  

soon as our negotiation recommendation package is  signed off by DIR executive management during in   docusign which we should get sometime today. It's  routing as we speak. Okay. So vendors that moved   into negotiation will receive the kickoff package  in the mail with a specific deadline for response.  

We are going to be strict to that deadline.  If we not received your kickoff email response   according to the instructions, we will cease  to negotiate. Okay. There are 155 vendors.   We need to move quickly. We have a lot of work  to do to meet the deadlines for this project.  

Therefore, we cannot allow lack of response from  every vendor. We don't have that kind of time. So,   be diligent. Be sure you are getting DI emails  to make sure that you responded on time. Okay.   Vendors are going to be lined up to the  negotiation schedule based on the date and time  

that I received their kickoff response email. And  I will have a chart and I will let you know. Every   second counts. No matter what time you are, you're  going to be organized in the order of your kickoff   response. Okay? And later on, management will have  to approve the negotiate schedule. And I will let  

you all know what that is once we are done and  we start negotiations. Okay, next slide. Well,   I made it short than an hour. I want to thank  everyone that is in this conference. I want to   thank you for your time and your diligence on  pro making your proposals and participating in  

the DIR solicitation process. I want you to know  that this entire project was not a one-man show.   It took over 30 people to complete everything we  had to do in a short period of 60 days. And we are   on schedule and we are hoping to have everything  done and to you by 5:00 tomorrow. And thank you  

for your time. And please let me know via email  if I can help you in any way. But keep in mind,   every minute I spend answering emails and taking  care of things that you already have answers to   in our documentation is time that I could be  negotiating contracts. Okay. Thanks again.