America’s Seed Fund Week 2023 | Inside the Head of an Evaluator

Sam good afternoon i'm eldon hawks partnership and innovation specialist here at the small business administration and welcome to another exciting day of america's seed fund as we begin i hope you've taken advantage of all the exciting events from this week including yesterday's agency open houses today we're going to dive inside the minds of those who review your applications we're going to pick their brains to see what works what doesn't and what can get you funded to better help you understand what exactly goes on during the evaluation process we have a panel of notable agency program managers that have years of experience in sbir and sttr to further illustrate exactly what happens during the evaluation process and what evaluators are looking for so first off i like you guys to introduce yourself i'll start with april so introduce yourself your agency and what are or is the biggest misconception you hear about the sbirsttr programs from applicants.

I'Ll start with you april awesome thank you eldon yeah my name is april richards and i run epa's sbir program the environmental protection agency and i have been doing that for many years we are one of the small sbir programs so the beauty of that is that i kind touch every proposal and every award and every part of the review process so i do know it i do know it intimately so happy to share that knowledge with you the biggest misconception i think i'll just take that question i'll answer like a politician i'll just say something i want to say but that's not really an answer to your question but for epa with a small program and their other speakers kind of represent larger agencies so i'll focus on the smaller agency perspective we really care about technical merit we really care about commercial potential and we really care about impact to the topic area so those are the three things that are the most important criteria that we're going to use when we look at your proposal that you submit to epa thank.

You abel april excuse me mina all.

Right good morning good afternoon and thank you for having me my name is mina khalil i am the us socom sbir and sttr program manager biggest misconception about sbir and socom there's a couple of mines so the first one is called the valley of death i believe there is and not just me there are others and that was spoken by our acquisition executive and softweek a couple of weeks ago there is no such thing as a valley of death there are multiple valleys and we got to cross through each one of them the idea is if we need your requirement if we need your technology to fulfill our requirements we will guide you to get through each one of those valleys you know to the to the mass production level but you know you always have to work with us and meeting our requirements the other one is you have to know somebody in the government for you to get an sbir award not true i think eighty three eighty three percent or so sbir so comes sbir has has been with non traditional small businesses names that we never heard before and those are the ones that never got you know never received any kind of a socom sbir or a socom award in the past and those are the ones that we look for we want to make sure this is a seed fund for america's small businesses.

Thank you mina now let's blast off the outer space with quintin.

Hey hey i'm also excited to participate in another session with sbr and the small businesses on the line i run nasa ignites sbir program so we have a brand new program that's more commercialization focused we look for firms that's looking to commercialize the technologies as eldon also mentioned i worked in the mainline sbir solicitation for many years well for several years and so i have a lot of knowledge in both of these different programs i think it's so many different misconceptions you ask a really good question eldon i think one misconception is that you have to write to the maximum page count so we give a maximum page count for our phase one and phase twos and we see a lot of firms writing to that that can actually hurt you because there's so much fluff in your application what i recommend look at whether it's i'm not as familiar with the other agency programs but look at what they're asking for in this listen for us we have subtopics and scopes look at the subtopics look at the scopes what are we asking for and be extremely responsive to that i think that i'll leave it there for misconceptions.

Thank you quinton i think that's a very good segue into our next topic and i will say this quentin you make a very good point as far as you want to make sure you are as specific and detailed and direct as possible with your proposal if it's ten pages twelve pages don't just write ten pages just because of course don't submit to either so.

You know tell your story talk about why you can be responsible so you know i guess what myself and eldon are saying don't look at the solicitation from a robotic standpoint look at it as you want to tell a story and how you.

Can meet that need exactly and that's a very good point and before we get to the evaluative process we got to talk about preparation proposal preparation and that page count and writing to what the solicitation asked for is very important and april i'll start with you with epa and even sbir in general are there roadmaps or checklists that can help applicants navigate through the technical requirements yes.

Yes yes yes yes there are a few things you have to do to submit a proposal to the federal government right this is a non dilutive investment it's great you don't have to pay it back it's got so many pluses but it is the federal government so there are a few things you have to do so you have to start early definitely start early look at what registrations are required right do you need a sam registration is it a contracting agency you have to register with sba you have to have a duns number or uei these new numbers and then for us you have to register with our application system so yes i will say for epa yes we have added a checklist and absolutely use that it's exactly what our contracting officer will use when he looks at proposals we do like a responsiveness review when they come in to make sure like they don't exceed the page limit that the budget is not over what we can allow so you check off everything on that checklist that's exactly what what our contracting officer will use to get you through that gate so we are really hoping that that is helpful to small businesses to navigate all the many requirements that.

There are thank you april mina is there a checklist for applying to the army or socom excuse me yeah so.

For socom we do have have our instructions and i do see at times that some may not read the instructions and just they submit based on their knowledge the instructions that we have for socom under the dod within the dod excuse me defense sbir sitter mil or defense sbir sttr mil we give an exact list of everything that we require how you'll be evaluated based on and what we use it for so please read that read that very diligently don't just scroll through it and press agree as we do to you know your your iphone or android you know user agreement right and we made it very very clear very very structured so please use that in building your own checklist building your own matrix to make sure that your your requirements make sure it meets all the requirements again it's it we get a small percentage of companies or proposals that do not meet those first checks those are this is the first gateway and and it is one too many it's one too many that you put too much effort into that we will unfortunately not be able to proceed forward with or evaluate technically just because it does not meet those those those requirements.

And i was muted quinton is there a checklist or anything like.

That for nasa yeah exactly so i'm going to talk about again i'm talking about two different programs right i'm talking about the sbir mainline solicitation and sbir ignite solicitation just as my colleague april said if you open up our solicitation it's clear as eldon just mentioned it's a checklist and it lists all of the different forms i'm specifying the two solicitations because with our mainline solicitation and ignite we're about to switch to a different platform our old platform ehb some of the things in the checklist wouldn't let you go to the next step until you've got until you fill those forms in but a quick tip make sure that whatever you fill in is relevant we've seen people fill in information from a previous proposal we know sometimes you repurpose things so make sure what we're asking for that even if it's electronic that that's exactly the field that you're filling in so go back and check even though it's electronics for sbr ignite this year we also have a checklist but it's going to be semi automated so some some of the information that you're going to submit you're going to submit electronically some of it you may have to upload a form but most of it is going to be electronically yes both both solicitations clearly lay out everything and make sure if we tell you this is the pdf form so for ignite we're going to say hey here's a list of pdf forms to fill out here a list of forms things that you may have make sure you have that one more thing is that most agencies are adding this kind of like foreign ties foreign national background checks that's you will see that in our solicitations for sbir and if you and we ask you to fill out that form because i know some firms have applied before that may be new to them make sure that you don't miss that one.

Thank you quinton i'm gonna stay with you for this one now this program is all about innovation and you know all four of us have been in the program for years now we've seen that shift as far as what is innovative or what isn't partially that's because of technology changes but more importantly the needs of the agencies change from year to year people or agencies looking for something different and what they need and what their needs are so how does one in their application adequately address innovation in their proposal i'll start with you quinton yeah that.

Is an excellent question and i was i was hoping someone would ask that because you know this is called inside the head of the evaluator well i will take a second and get inside the head of the proposals okay and i'm going to answer the question around about way if you take a picture proposals who is the first person that you're going to look at in that picture yourself right just the first person you're going to look at it so from a nasa perspective if the picture was taken look at it as if you were nasa and what is nasa working on at the time obviously we're working on artemis we're working on moon and mars everybody knows that however in our solicitations we have subtopics and we have scopes look at our scopes we update them every year just like eldon mentioned according to how technology changes according to what the agency needs and be very responsive to those scopes i don't know i probably talked to this before like i've written tons of proposals you know i started writing proposals in graduate school i didn't have the best gpa so i had to write these proposals to go to grad school i eventually got really good at it and one of the things that i that when i was not winning these these awards i realized that sometimes i was trying to take what i was doing and stretch it a little bit because i wanted to go after all these different opportunities yeah i really needed the money i didn't have scholarships like eventually i got scholarships because my grades were good but i was trying to stretch i was trying to make something fit where it didn't fit whereas these organizations nasa included i've been rejected for nasa awards pretty much most agencies i've written proposals for as a grad student so that's kind of my advice is that make sure your technology is a good fit and don't try to stretch it.

Thank you quincy april how does one define innovation for the epa.

Okay yeah sorry again eldon i really want to talk about the other topic that quentin was building on just responsiveness especially for epa for environmental protection agency small agency we have fairly specific topics unlimited topics every year because we just don't have that much money in our sbir budget so we're really looking for proposals that are responsive to those topics so just like quentin said we really want something that's going to help us solve those problems that we identified our mission is to protect human health and the environment on this planet we're worried about this planet quentin's going to get us to some other planets but we're thinking about earth and so we really epa two two of the reviews that you get are going to be epa people so if you can convey that enthusiasm and impact to environmental protection through your proposal that is really going to be a big pl in getting reviewers excited about your innovation.

Let me ask you a question based on this so would it be safe to assume that being or specificity and being very specific to what's being asked for is a very good way to convey the innovation of what someone's proposing to epa i think that's true.

We definitely kind of know what we're looking for i mean we know what the problems are let's say we always try to define the problem you give us the technology solution we're not telling you what technology but we've got air quality issues we've got water quality issues you help us solve those.

And even.

Just more process wise like we evaluate for innovation and ip through our commercial review in phase one so we're really looking to see that you have thought about intellectual property and how you might protect that like just to dive into the ip side a little bit okay so i'll leave it at that for.

Now all right thank you april mina could you talk about how someone demonstrates their innovation or the innovativeness of their technology to socom yeah absolutely the innovation.

For socom should not be that hard because we take right out of our right out of our requirements and develop those topics that we put out and because of that we want to see a solution to what our requirements are we want your your technical team to write that that proposal we have a very good solid technical sme's that are able to able to evaluate this and see exactly what how you guys are developing the work so you know stay away from the from the fluff you know fluff is good not so much right just just kind of minimize it we want to see your technical aspect how are you going to solve the problem how are you going to deliver that prototype what is your end goal there how are you going to commercialize it what you know your technical merit your your team members who is going to work on it and the resumes because we do evaluate based on technical merit on personnel on a commercialization plan we want to see you successful i want to see your tool in the hands of our operators so that's the innovative part now that said maybe there are some aspects in that one pager that may not be that clear we have virtual q and a's with for every single topic that we baa that we have out please join in ask the question start early ask for the questions don't come in the last minute and propose something that is entirely fluff or missing information or a part of the usually a part of the solution doesn't cut it usually we want to see a full solution to what we have but please demonstrate it through your technical technical merit of your ability to.

Do so now mina you and april brought up the process so let's jump ahead just a little bit and i'll come back to another key part of that which is going to come up in this so you talk about the review process what is socom's review process and what are reviewers specifically looking for when they review applications yeah so from.

The socom side and our review process again it starts with the with with the it starts with the not with the responsiveness part make sure that we have everything that we already discussed that and then it comes with the technical team and technical team looks are the ones that author the topic so they know exactly what they're looking for and they know what the technology is all about and that's when that's when they start you know that's when we start evaluating every single one to see can do they actually have the people to do it and is the requirement and technology that you guys are offering is that going to meet what we're looking for and then at the end are you going to be are you going to exist after at the end of the siber phase one or phase two or are you going to say sorry we are in a you know we locked ourselves up as a small business in a monophthasis of a monophthasy of socom to the us government and there's nothing else outside of that and then looking at this whole picture picture then they they come up with you know we come up with the with the the final criteria of saying here are the qualifications at times we do require we do require now our specific topics fifteen presentation slides to say hey we have five qualified that qualify we want to down select to four or three and then we have a one hour session where we have that discussion with with the company and make sure that they are actually doing or able to answer some of our questions from the technical side and that's when our final final selection we don't do our selection based on a set number of phase ones and set number of phase two it's more of who can actually do it technically who has the right people who can commercialize and then based on that on that we can make a decision of of how many we need to do we just want the qualified.

Ones right now me i don't know if i missed this but who are your reviewers are they come from the outside or they internal to socom itself.

They are all internal to socom so our topic authors are the ones are from the program offices and the program offices are the other ones are asking for the technology they and then obviously them as you know we have a lead evaluator and then the team itself the team itself could be members of the of the you know the operators the users themselves it could be support contractors it could be sme of some sort and we all we they have that that team put together in order for them to do it they could be from the program office or from our our socom components depending on who has the need for at the end we we do there is a small for phase twos we do require a an acquisition plan a transition agreement to be signed between the program executive officer or the funding office and our s t director and ses to say that there's a transition plan for a phase three for that so everything is lined up directly to say we're going to transition and that's why socom is one of the best in the us government in transition rate from phase from phase two to phase three or from a silver phase one phase two to a.

Phase three now before i jump to april because you you said something very could you explain to them what do you mean from transition from a phase two to a phase three because some people may not understand exactly what you're.

Saying there yeah so a phase one is a concept right you're you're writing a study doing a study you're saying how you're going to take this low technology readiness level this concept and you're going to you're going to build it as a prototype and everything that's a research in between and i usually is about six to seven months period of performance again you know it depends on the agency it depends on the dod component and then out of those we do select or generally the civil program office would select the phase two the phase two for prototyping so that's the prototyping phase and that goes it can go anywhere from eight months to twenty four months even longer depends on how new kind of cutting edge technology and the ability and the feasibility of doing so when we finally go through those phase ones and phase twos for sivir which is funded by the sbir sttr dollars with lim station that is set by sba then the program offices or commercialization picks up and that's when you can you have no unlimited contracting ability you're able to to commercialize it in the market you have no you know you have no cap on how much funding you get and i've seen companies that started off with one hundred fifty thousand dollar phase one got a about a million and a quarter or a million million a quarter so phase two and then boom they got a forty nine million dollar indefinite or contract over the next few years to to go ahead and and fund more so that's the phase three that's your success that's you putting this stuff out and that's what grow that's the growth that will take you from a small business to something to what qualcomm is now because qualcomm was a phase it was a silver awardee back in in the nineteen eighties and look where they are now.

Exactly and all of our devices so i thank you because we got a question from the audience about using the technology and just sort of reiterate we have granting agencies as well as contracting agencies and one of the things you'll see as we go back and forth the evaluation processes are different but more importantly contracting agencies there is an option for that phase three that phase three that commercialization where you get the socials contract many times granting agencies they don't typically purchase the technology once it is created because that's not their goal their goal is to advance in their research efforts not so much to fund a product to be commercialized for their needs and that's something you're going to see as we go back and forth amongst the agencies and i hope you learned yesterday at a lot of the open houses as well but i'm going to jump to april april could you talk to us could you gave us a little bit of a snippet about your review process could you talk to us about the review process and who the reviewers are for epa yes i would.

Love to talk about my review process and actually i love to just watch jack interpret our review process i've never seen it look so amazing interesting yeah so for epa again we are only the proposal so like mina was mentioning like they have a pitch or slides or an interview for us it's just a proposal small agency we don't have the bandwidth to do that so we're going to evaluate whatever you put in that proposal so this is kind of obvious advice but it's a good advice is address all of the criteria in the solicitation the evaluation criteria they're laid out you're going to get a score on each one and the higher your score the more likely you're going to be in that pool of being considered for funding so this is speaking for epa so you really maybe some criteria you're really like your team your technical team is super strong maybe your commercial team is not quite as strong but either way you need to address all of those teams criteria and get the best score you can for each one and that'll give you your cumulative score that hopefully puts you in that funding.

Pool.

For epa for both phase one and phase two we do contract out a commercial review so there'll be some external person looking just at your commercialization criteria and then it will be epa experts kind of like the military it'll be like epa experts looking at both the technical quality of the proposal and the relevancy to the topic how much it addresses the topic and how much environmental impact it will have okay thank.

You april and quincy could you tell us about the i guess in this case you have the ignite sbir ignite as well as the mainline program could you talk about the review process for those you know talk about similarities and.

Differences yeah so for the mainline solicitation similar to april's program you're evaluated based on what normally what's in the solicitation the solicitation hasn't come out yet for py twenty four so normally it's scientific and technical merit and you can just find this right in the solicitation i think it's scientific technical merit it's experience and qualifications and facilities effectiveness of your work plan and it's also a commercialization score commercialization score is usually for phase one we don't look at it as strongly because we know firms are still working on their commercialization plan our reviewer pool mainly comes from nasa subject matter experts again i'm speaking in terms of the mainline solicitation for the actual commercialization review aspect again phase one not as much but we want you to be thinking about it phase two similar to mina's program phase two we look at commercialization a little bit more because we want to again phase two we want you to be actually moving towards not just being dependent upon nasa awards and government agency awards and so those commercialization reviewers sometimes come from within the agency but we do contract them out as well so we have a team of people within the agency sometimes i'm even called on to other programs that have a commercialization understanding they have a business you know they have some experience in business but they also have some technical experience developing technology we usually utilize them for the commercialization review now for ignite we actually have a lot a higher pool of commercialization reviewers sometimes they come from the agency but most of the time i would say seventy five percent of the time they're external and these external reviewers could be investors these external reviewers could be people that are in the technology or space and commercialization space and the commercialization review starting at phase one for sbir ignite is much higher than the mainline solicitation because it's more of a commercialization focus award we also have nasa subject matter expert as reviewers for ignite so for ignite our review pool is made up of a lot of different people but they're experts in the particular topic areas that we have or they're commercialization experts or the investors.

Thank you gwen now i'm stay with you for this next part because we have a question here that's very key to the evaluation process and i will say just from from my experience and i have been a program manager as well as a reviewer at sbir during my previous tenure of my career and one of the things that i've seen after reviewing proposals after sitting on many review panels is that the composition of the team that you have in place can be make or break you can have the best technical proposal ever but the composition of your team could really be whether or not you get funded could you talk about how important the team that you put together is for that proposal particularly the pi yeah so.

This is a great point and i mentioned you know in the solicitation again the next year solicitation hasn't come out for ignite or for mainline solicitation but normally it's the experience qualifications and facilities section in that experience qualifications and facilities section we're trying to see you know is your team capable of doing the work and sometimes for small businesses we know that you can't do everything but that's where your partnerships come in right you can always partner with somebody who doesn't have the experience that you have or they don't have the facilities i know you firm said you know phase one is not a lot of money but you really want to utilize your phase one to kind of get your foot in the door with these agencies get in front of other people the money is one thing but you really want to use it for networking so even if you have to give up some of your money to network and partner make sure that team is strong because just like eldon mentioned that is a review criteria in our solicitation again look at other agencies solicitation see what's in their solicitation but for mainline solicitation and ignite we both have that experience qualifications and facilities and we want to know can they do the work right and that's going to be also reflected in the work plan so you know if you have the people if we come back and look at your work plan and you develop and you know i was a microwave engineer we see that you're developing some instrument but the person that you have as your pi has never done this before we're automatically going to look at it and say they can't finish the work we want you all to be successful so the pi at it's very important and i made that point because some people always say well if i don't have a phd that's why i didn't get awarded but that's not the case you know if if you if doesn't matter your level of education what have you done and have you shown that you can develop the technology even with a master's degree if you have tons of publications on google scholar or you have some part history you've been working in the industry you've been developing instruments again i was a microwave engineer you've been developing these instruments for this agency for this company we're not going to just say oh they don't have a phd we're going to look at can you do.

The work thank you now i was going to ask that phd question because that is a question that comes up all the time as far as a pi is concerned do i need a phd do i need a phd april i'm going to shift to you talk to us about what epa looks for when it comes to the composition of.

A team for a proposal yeah definitely so again we have those fairly specific criteria and we have one criterion that is the team being evaluated from a technical standpoint and then we have another criterion that's the team being evaluated from a business standpoint like can they take this technology to market so we actually kind of evaluate it twice from different perspectives so it is important but we have other criteria that are important too like do you have a plan to take it to market is it innovative is it responsive to our topic so it's all they're all weighed in there and the weight is in the solicitation so there is a balance and so i think the pi is important but i don't think it's do or die we do not care if you have a phd as long as you have expertise experience something bringing something to the.

Table.

And actually talking about experience and expertise i do want to say something that i was thinking about earlier like we really do want new companies in the program too like we really are trying to get startups and companies that are new to sbir so you don't have to have this enormous track record either like either you can have a track record or you can tell us your plan to get it to market you have a good plan this is how we're going to do it this is who we're going to subcontract to like quentin said this is who we're going to partner with to fill any gaps that you might have because we we fund a lot of like two person companies five person companies and we want that you just got to demonstrate as best you can in your proposal how you're going to do what you said you're going to do in six months for one hundred thousand dollars exactly.

As a really putting yeah mina how about you talk about socom and what they're looking for within the composition of.

A team all right so the composition of a team is a theme that can do the work with the right experience and the ability to produce so you know we don't usually say hey shall have a phd and if we if if we do require a phd chances i will put that in in the instructions and put that clearly in there but we never actually did that or not in my tune or not in the structure of the team can be and you know i'm kind of echoing what april just mentioned you know the technical team is important the scientists behind it who's going to do the work who is going to do the business aspect of it and again who remembers how many who remembers how many the three parts of socom how socom evaluates the technical merit the personnel and the commercialization plan so putting a solid commercialization plan tells us exactly where you're going to go after you know the phase one or maybe between the phase one and the phase two or after the phase two or how else are you going to grow outside of the the scope of just socom or just dod or just the us government outside.

Of that thank you man i want to i want to make this point to everyone to give you an actual real world representation of everything they said based on my experiences when i reviewed before and i remember there's one proposal i reviewed they had great technology the technology is really good i start looking at the background of the team and they talked about at this point it was about this is five six years ago using cloud technologies no one on the team had any sort of expertise or understanding and either using azure aws any sort of the major cloud technology that were there and didn't demonstrate that in their body of work you can't say you're going to do something or able to do something but you don't demonstrate in your team that your team has capacity to actually deal with the technology and make it work and that's you know a very specific example that i remember because it always stuck out with me so you want to make sure you demonstrate that your team has the credentials and capacity to do the work so it's not just if they have a phds like can they do the work and are you demonstrating that in your proposal now one of the things i wanted to ask about and i'll start with april because this came up when you were talking april particularly with commercialization i'm looking at phase one we know that phase two it's very big part of the application process and the review process how big is commercialization in phase one for epa.

It'S a third we literally have a third a third a third so we have a third for commercialization a third for technical and a third for responsive so it is important i don't know my philosophy is we're a small program we need to get thinking about commercialization right off the bat you can't just love your technology and think it's going to be great we need to know who's going to use it maybe you've done some customer discovery some thought about who the end user is or how you would get it to market so it is pretty important for us thank you quentin.

I'Ll shift to you.

Yeah so again two programs mainline and ignite mainline we want you to be thinking about it and so what that means is i'll just pretty much echo everything nero and april said you know do you have a plan have you thought about it have you started talking to customers one of the things that i wanted to give firms a tip on on this call go through i corps even if you know we don't have one at our agency just find the icourse program to go through that helps with your commercialization plan and it actually helps you with your proposal because you start to develop something somebody needs it more than likely the government government may need it as well with regards to ignite commercialization is huge you know it's huge in the sense that we choose our topics to determine based on do that is there a commercial space or aerospace market meaning they can you get commercial space customers from big companies small companies your lockeys your boys your raytheons can you give follow on investment are there investors like sometimes you don't have a technology that's ready to go to market yet but you have some investors looking to invest in that technology right now so you know can this topic produce technologies that investors or other government agencies are going to want to utilize or invest in this technology right so all of these different things of commercialization we choose our topics based on that so obviously we choose our firms the same way when we look at the application we're looking at what is the total addressable market what is the service addressable market meaning that can you take this small part of the market this kind of goes back to the evaluation as well because we want to know have you looked at your competitors anybody that started a business just because you do something a different way doesn't mean that that we're going to buy your technology have you what what is your key enabling technology so this kind of goes back to your question about evaluation elder so a few a few things that i look for different people look for different things right what is the current state of the art do you improve upon the current state of the art from a technical perspective and does your improvement on the current state of the art really address a nasa need that same technical aspect of current state of the art do you improve upon it you can apply to the commercialization section right like what is different about your technology will people purchase it right and what is different from your competitors and so so these are the types of things that we look for in the particular ignite solicitation again we also look at it in mainline but just not as.

Much thank you quinta mena could you talk about that commercialization aspect specifically in phase one and how it's really looked upon and evaluated by your evaluators.

Yes so let me make this clear no one is safe from the inability to produce a very good technology and in in my previous positions capacity i've had a major defense company that produced a technology a system that can do exactly what we're looking for just can't produce it so thinking up front and saying if i'm the phase one when you're writing that when you're writing that study and saying you you you shall you have you you know your your life your company depends on not your life your company depends on your ability to to produce that system to commercialize that system is key is key to your success because then if you develop a software that cannot be and i know a lot of people will say a lot of companies would say well i do i do software that's easily produced but if you produce a software that cannot be plugged for loopholes cannot be protected from cyber security then that system is not useful for us if you develop a system that you cannot you cannot build the you cannot build the production line for you cannot produce then that system cannot be used and all of that starts in phase one if you if you develop something with supply chain problems supply chain risk risks you've already developed you've already developed a risky system that cannot be produced all of that should start in phase one you know your ability to do this as a cutting edge technology or how far ahead there's a very good example in the past on what is it the the the current bomber i i believe the b twenty b twenty one i can't remember and when they started building that upfront years and years ago before software was was was able to to manage the airplane was just wings but they couldn't fly it because there was no software that can do it and you know when the time came and software is able to do it then that's when that's when when that system that the airplane was able to be built and and it became something very important in our military likewise when you're building something and you have a technology that has we're kind of ten years ahead and not there yet how are you going to to commercialize that system so keep that in mind as you're building the system and then and then finally and i'm losing my train of thought here finally with your commercialization think and as quentin mentioned and april mentioned thinking of that two use for your for your prototype how are you going to sell this system to the government and sell the system outside of the government what is the how are you going to commercialize it in socom we actually we we worked very closely with our partner intermediary a non for profit called softworks and we've put together a small business boot camp where we teach them a lot of how to be you know how to work with the with the government etcetera and it's free and then as part of it we brought investors venture capitalists to sit in there and talk to the small businesses how they can utilize some of that funding outside we either put some small business cber awardees in front of those in front of those panels and say hey pitch what you have and see how you can commercialize it outside and it's very interesting how some of the weak points were that they couldn't commercialize commercialize their technology outside of the us government and that is a concern thank.

You i want everyone to really take stock in what everyone has said and really to understand how even in a phase one how important that commercialization piece is you're going to have differences amongst all the agencies but i've been in sbirsttr for the last eleven and a half years now there has been a decided shift in the level of understanding a phase one organization needs to have about the potential commercialization of their product that has become more and more across the board regardless of the agency something that is looked upon even more when it comes to that phase one it's not just about i have a great innovative product or service it's about can this be commercialized and do you have an understanding even in phase one of how it can be applied and i think mina made a very good point with that especially since they do contracts you want to look at at what's the government application but also what's the application outside of the government as well if there is one so it's very good points i want to go back to just the team aspect because we had a very good question come from the audience and we'll go a little bit further as time begins to run out so we talk about the team the composition of the team does the team that you propose have to be working on that proposal or can you pretty much say this will be the team if you get funding i'll start with you meaning since we just finished.

So please restate the question we're talking about the size of the team or.

Do you have to have the team in place at the time of proposal.

Yes yes so so so this is a very good one because we came across few of those in the past where a proposal came in and it had some solid technical aspects of it and then it talked about the team size of so many but then when we had you know we had our questions obviously when we read this the the the the those proposals and we have the background we have the technical background we have the experience we've had some further questions and when we talked with the company the company was not the team that was mentioned there was a company of one and everybody else was i'm looking for a hire of this i have someone for this i'll be working with i'll be working on getting this if i get the proposal and that is a concern if it is a team of one tell us is a team of one and it really depends on the kind of level of work that you're trying to that we're trying we're looking forward to see if that actually fits in within within your your your your team size so we need to know all of the stuff up front because that meets our second criteria in the evaluation of the personnel working on the proposal they're not the they will be that they are who they are now okay april how.

About you does the team already have to be in place at the time.

Of the application yes because we're basically evaluating what you put in the proposal so yeah we need to see who's.

On that team okay okay quentin same.

Question yeah same same as mina and april we have had situations where during the award sometimes between the application and award team members change if you award it there's a period called contract negotiation and if team members change you'll have to submit talk about that or work that in a contract negotiation period is not guaranteed it can be denied because again just like my colleagues said we evaluated your proposal based on that right.

Now one more question about teams and again this is very good it kind of goes back to the whole phd question we have a question from the audience what would you suggest for an owner of a small business who does not have a business background in technology that he's created but has a strong team let me ask does that question does it come down to who you select as the pi or is it really about the team regardless of who the shepherd is for the project and i'll start with you quinton yeah i.

Think we're going to look at the overall experience of the team i think for nasa's sbir program you really want your pi to be the technical person you know and i think it makes sense right because that's the person that's going to be interfacing with the core we have we assign you a core contract officer representative but it's all that's that course also a tech monitor that person is going to be interfacing with the tech monitor so we want we want that person to be like an engineering lead or something like that for.

Nasa'S program got you april how about you.

Yeah i don't have a lot to add there again we're sort of looking holistically we just want to balance that you've got the business side and the technical side covered so i won't spend too much time on that i.

Think i ditto okay mina all right.

So good question our technical poc or core as quentin mentioned is going to be the one interacting with your primary investigator at several times and i've seen and i met with companies where the ceo is not necessarily the primary investigator and rightfully so we do have multiple contact information we have a primary investigator and we have a corporate official primary investigator is a technical guy that's the one that's going to do the work and the study and manage your team.

Okay thank you mina and i have a question you know you've submitted the application you've been reviewed you didn't get funded do you get feedback from the agency based on your decision let's start with you mina.

Yes and please please please take full advantage of that ours is elective other dod components make this you know a standard thing and thanks to our if you're not familiar with the dsip tool it's or a dsip website it's dodsbirst mil that's where what is it two to three billion dollars of dod of sbir out of the entire sbir budget comes out of dod that's more than fifty i believe and that website www dot dodsbrstr dot mil is where you you certify who you are small business provide everything and then you get to see all the topics you get to ask questions about the topics and you get to propose to those topics and at the end you know with with the current progress of the development of that of that website or evolution of that website now you can get your informal feedback or your feedback directly from that website so you get to see those for us in socom it's by by request about seventy percent of our proposals that do not get that are not selected do request that feedback and please do so i've had four or so small businesses present and soft week a couple of weeks ago and talk about their experiences their successes their challenges and some of the things that they've said is hey don't just you know give up because you didn't get the very first sbir proposals awarded keep going and eventually you hit them and how you hit him you get those you get those feedbacks because that's how you tailor and adjust your proposal to meet to meet to meet the criteria that we're looking for thank.

You mina april what sort of feedback.

Does epa give yeah absolutely just what mina said we do provide feedback you just have to request it so absolutely take advantage right you're going to get expert feedback free advice from our commercialization experts from the epa experts so yes it's a resource and i will say we don't fund everybody we want to fund so just because you didn't get funded doesn't mean you don't have a good idea it just means we did not have enough budget to get maybe to your proposal or something so i don't feel like i need to tell entrepreneurs that no doesn't mean no you should be persistent and keep plugging away get what you can and then keep resubmitting but definitely take advantage of the.

Feedback okay quentin how about nasa so.

For both programs we do give feedback but with a caveat some proposals are screened for non responsiveness meaning that they were not responsiveness this goes back to the subtopic or the topic that you're writing to those don't get feedback but the rest of the proposals get feedback and i ditto everything my colleagues said take advantage of that feedback okay now.

I will say this from just from personal experience and what i've seen over the years when it comes to that feedback now full disclosure before my time at sba i was at department of agriculture working with the sbi program and one of the things that i saw and this is goes to all the points that they made the feedback that's given it's free advice it's free critiques from individuals whether it is internal to the agency who know exactly what they're looking for or in cases like say usda or the granted agency they're using external viewers where they are subject matter experts and nine times out of ten when agencies excuse me organizations businesses innovators utilize that free feedback they come back with a stronger proposal than next year and i can't say you know fifty percent of those that reapplied the following year were funded in my experience but over almost eight years there i saw a very good number of those that reapplied and utilized that feedback ultimately whether it was year two or year three ultimately had a very good chance of at least not getting funded or at least recommended for funding as well so that free free feedback can really go a long way between you know you have a very mediocre proposal to having a proposal that is worthy of being funded and could really be something that the agency could use down the line as far as your technology is concerned we're just about out of time i'm sorry i didn't get to all the questions but i'm going to start with april first could you give any last words of advice on giving a good impression with the proposal yeah yeah i.

Was just thinking about that i think my biggest advice is kind of building on something we started with about how some people jam everything they can into the page limit i would like step back and have someone else read your proposal like make it readable and understandable sometimes you're looking through that and like oh my god look at all this data and all these facts and i'm like i have no idea what this technology is going to do like i can't quite tell what the impact will be so i would say maybe hand it to someone that's not too deep into it so and see if they can understand and convey that excitement and impact about what your technology can do.

Thank you april mina how about you.

I'M going to kind of end with the same way i started please read the instructions of what we're asking for please stay away from the fluff you know we want to see the technical piece of it we want to see how you're going to to prototype you how are you going to develop that new technology and we want to see you succeed so that's my biggest piece of advice when it comes to writing and developing a proposal quentin how about.

You yeah i'm going to combine what both of my colleagues said april struck a nerve because she reminded me of the first time i won a nasa award so before i worked for nasa remember i was writing proposals for graduate school and all these different things my first and this is like coming from double digit rejection so i had written tons of proposals i started to get kind of good at it but i had never won a nasa proposal just like april said take it to somebody else i took it to a guy in my department i was in grad school so i was much younger so we called him shake daddy his name was doctor shaker banzali he had not seen my proposal i took it to him and he looked at it and this is so awesome he looked at he said okay what are the requirements in the solicitation same thing mina is saying he looked at the requirements in the solicitation he took my proposal and he made all of my headings the requirements in the solicitation so if it was an sbir solicitation experience qualifications scientific technical merit facility work plan right and if it was ignite commercialization he took it and he made those dissections i didn't have to change much because i had the material and that was my first nasa award and from that point on that will also answer your question elder what would be a tip from that point on that's how i started writing proposals you look at what they're asking for i'm not saying leave anything out let me put a precursor this is not mandatory this is just a tip you can take the advice or not but set up your sections with what's in the evaluation criteria because the reviewers this is inside the head of reviewers we're looking for what do we ask you for in the solicitation for the evaluation criteria so if you make it easy we can go and see do you have it now doesn't guarantee you're going to win because especially for nasa it's super competitive so this goes back again to what mina said is that can you be super responsive right another tip that i want to give firms that i already gave you is get into the icorps program you know if you can before the solicitation another tip we have a period that we don't have an open solicitation we're not in blackout reach out to people at the different agencies i can't really speak for epa and i can't speak for socom but for nasa i think everybody does it you could ask them hey can i speak to a subject matter expert send a little blurb don't sell it don't send me a sale page saying and you know we can develop this but you know talk about a little bit about again what's the key enabler like what can you do different how does it how did it help nasa science no more than a few sentences and we'll try to connect you with a subject matter expert so those would be the tips that i would give for firms that's applying thank you.

Quinn now quinn can you let everybody know the best place to contact your email website all the details.

You'Re muted when.

All right sorry about that i i live by a metro so it was kind of loud but just look at our website sbr dot nasa dot gov and just google nasa's sbr program again i'm not the main i'm not the program manager for the mainline solicitation that's dina dina salazar and you can you can reach out to you can find most of our list just go.

To our web page thank you gwen april how can everyone reach you and find out more about epa's sbir sttr.

Program absolutely same thing google epa sbir takes you right to our homepage all the general information there and i just have to say we have one solicitation per year it's going to open next month and we'll release the topics next week so if anyone has an environmental technology check out those topics and see if you can help us with any of those i am kind of it for epa so just email me richards dot aprilpa dot gov happy to like you know bounce ideas if you think you have a technology that might fit happy to give you some general advice about whether it's appropriate for solicitation thank.

You april mina how can everybody reach you and learn more about socom's program.

All right so for us we have a www dot socom mil and then under doing business you find socom that website sucks and it doesn't the siber one right not not the entire website just the siber page it's it doesn't have anything except some ancillary documents now from another alternative is we have multiple other gateways to socom something called engage soft it's on this doing business under the same website that's how you can submit some ideas and let our technical pocs see the stuff or you can go to softworks dot org softworks sof w e r x dot org they have an ecosystem they publish every daa that i put out and they put all the documents in one place they hold all my q and a virtual q a sessions for any of the topics and that's a great way of of asking questions and finding out things firsthand from the authors of the topics for me the the email is sbir at socom mil that one gets all my teams so if i cannot get to the email somebody in a team very very small team team of three are able to to answer or get it to my attention so that has a better attention than my personal email linkedin is a great way of quick chatting and talking and when you send me a linkedin or sbi or cberocom mil i can easily send you a calendly link and you can just schedule a quick fifteen minutes and we can have a further conversation so opening a lot of these doors one last thing is socom our acquisition technology and logistics acquisition executive under our office of small business programs also on socom mil i'm throwing a lot of information here they have something called an acquisition executive roundtable it which is every other month it's free any small business can participate and you can ask any questions not just for sbir sttr but for socom as a whole definitely encourage you going in there you have a plethora of different sme's sitting on a table to include the acquisition executive and a contracting and a small business et cetera and myself engaged soft et cetera that would answer all your questions so please communicate with us talk with us we'd love to.

Hear from you mina thank you april mina quentin thank you very much for your time i'm pretty sure your linkedin is blowing up right now i appreciate it all and all the insight and information that you guys have distributed to all of our wonderful attendees to use the deck.