America’s Seed Fund Week 2023 | Is America’s Seed Fund For Me?

Foreign.

Hello everyone. I'm Nancy Kamay from the Small Business Administration and we're here today with three esteemed panelists for our panel is America's Seed Fund. For me, the goal of today's session is to get innovators, founders and others who support them thinking about identifying and articulating an important project at your company that will be of interest to an agency to fund. Also, we hope you will glean some concrete next steps on your SBIR STTR journey. Let's start with introductions. I'll call on each of the panelists to introduce yourself and give a very brief introduction to your program. In particular, can you please talk about your program's focus? What types of projects or technologies is your agency looking to fund? And please address whether there are specific deadlines or not. So let's start with Ben.

Thanks Nancy. Appreciate the opportunity. My name is Ben Schrag. I'm a program Director and policy liaison in the Small Business Programs at the National Science Foundation. I've been at NSF for 14 years and was prior to that at a startup which was also a recipient of funding from the NSF SBIR program. So have been PI as well. So to give a quick summary of NSF's program, I think the key things to keep in mind, number one, our focus area really is rather than a topic or a specific type of solution, we are really looking to support pre seed stage deep technology startups. Our focus area is kind of a stage and type of organization. We are looking to in most cases be the first funding into a company at the stage when it is too early and too risky for private capital. And so the way to think about NSF is as kind of a strange form of investor that is only going to pay for R and D and that is looking for high technical risk. And so like all agencies, we are looking to fund risky R and D. We probably even want more risk than many agencies. So a lot of times we will be the first agency that a given company will be appropriate for. As far as technology topics, the NSF is basically as wide as you can imagine. There's no technical area that we don't support. The only kind of thing in the broad spectrum of R and D that we don't pay for as a rule is clinical trials. If you're getting down that path of FDA approved things, Stephanie will talk and maybe she'll have a different answer here. The other thing to point out about NSF is that I think one of the differences and the advantages of our program is that it is led by full time program directors who make the funding recommendations and all of us have both technical backgrounds and also have experience in the startup world as either founders or investors typically or have otherwise had private sector roles where we brought technology to market. Meaning we've been in the shoes that many of you are wearing and we have a lot of empathy and a lot of admiration for the universe of technology entrepreneurs.

Thanks Ben. Can we move to Susan.

Hello, Good afternoon everyone. I'm Susan Sellis. I am the Director of the Defense SBIR STTR Program Office. I have been with the Department of defense for 35 years, 25 of those years being with the SBIR and STTR programs. I've been in this position at the Office of the Secretary of Defense for Research and Engineering for four years and our office primarily oversees the policies. We facilitate communication, collaboration and coordination across the 13 DoD components. We also influence legislation and interpret policies. The other thing is we have our own program. So OSD has its own SBIR and STTR program as well as a TRANS program. The Department of Defense is interested in all stages of risk, so high, medium and low. It just depends on the needs and the missions of those organizations that are participating. We do a combined hybrid of centralized and non centralized management of the programs. My office puts out all the broad agency announcements which includes all of the topics for all the participating services and components. We do have deadlines. I highly recommend that you sign up for the listserv so that you will get notifications of when solicitations are opening, closing and the pre release period, which is very important for DoD. It's the most important part of the RBAAS because that is when companies can have direct contact with the topic authors ask technical questions only about what they see in the topic. If you need clarification on anything, that's when you can have direct contact. Once the BAA is open, you can still ask questions, but then that's posted for everyone to see the question and answer. So many times companies don't want to give away their approach during that time. So the pre release is really the best time for you to ask those questions. And our BAA is actually going to open tomorrow. We have a SBIR and an STT opening tomorrow. So get your proposals into us. And Nancy, are you going to have, are you going to provide slides or to the group? I know I had a slide deck, but I guess because of the time here I didn't use my slides. But it has a lot of great information. If you're planning to share that with the attendees.

We will work on that. Susan but at least there is the one pagers and videos that people made that's available in the resource area.

That would be great.

How about shall we move to Stephanie Great.

Good afternoon or good morning depending on your time zone, everyone. My name is Stephanie Fertig and I'm the HHS Small Business Program lead. I'm located within the National Institutes of Health in our Small Business Education and Entrepreneurial Development office or seed. And here at HHS and nih, we're certainly excited about being part of America's seed fund. There are four components in HHS that have an SBIR program. Fda, cdc, acl, which is the Administration for Community Living and nih. NIH is the only one of those that also has an STTR program. Now, our mission can be summarized as turning discoveries into health. And our programs take the great innovations across the country and get them into the hands of the patients, clinicians, caregivers and researchers that need them. We tend not to be the final purchasers of the products that you're going to develop. So unlike the DoD or other agencies, we're here to help de risk the technologies by supporting that research and development and get them to an inflection point, usually where either a partner or investor is going to help a company take that innovation to the marketplace. Some rare cases that might be enough to take something to the marketplace, but oftentimes it takes a partner and investor. And we recognize that we're in the early stage, but we also have a lot of flexibility. And I think that's one of the big themes of our program is flexibility. We certainly support very early stages, but has been alluded to, we can also support clinical trials. We have three standard receipt dates a year and we even provide flexibility with regards to funding. While there is the standard SBA budget guidelines, we do have a weaver to exceed those guidelines for some topic areas because we recognize that when you're talking about health research, it may require more funding and more time than the standard SBIR guidelines allow. Now, the vast majority of what we do is grants and the mass majority of what we fund comes in through our general or omnibus solicitations. And that means that these are investigator initiated projects. These are great projects where you as a company have identified a problem and are coming to us with a solution. And that's a problem. Within our mission area. We do have some contract. We do have some contracts and we do have some specific notices of funding opportunity. But again, the vast majority of what we receive and what we fund. Come in through those general omnibus solicitations. Finally, I would say you take nothing from what I say today. Come talk to us. Talk to us in advance. Unlike what Susan just said, since we predominantly do grants, you can come talk to us at any time. Please talk to us in advance of submitting an application. If you're not sure who to talk to, you can start with our website, seed.nih.govsed.nih.gov. you still don't know who to talk to. You can always send a brief description of what you're planning to propose to seedinfoih.gov and we can help you. But please come and talk to us.

Great. Well, you all kind of addressed what was the next topic is about reaching out to an agency and finding a fit. And I'm just going to kind of take the position of the voice of the innovator, of the founder. So say I have a project that I think your agency would be interested in. Is there a way for me to assess that, whether your agency is interested or not? Like, what should I be reading? Who exactly should I be calling? And in general, is reaching out to your agency something that you recommend? So let me go back to Susan.

Yeah, so yes, absolutely, we recommend you talking to us. I mean, events like this, the road tours, the different outreach events that all of us have various organizations. Again, that's why I'm recommending signing up for the listserv, because the Air Force has asked me anything. Webinars, the army does webinars, everybody does some kind of outreach to answer questions and help guide companies with responding to our solicitations. So we do want you to communicate with us. And sometimes some people want to know about the timing so you don't have to wait to see a solicitation in order to kind of get a feel for whether we're interested in your topic or not. Look at some of our past solicitations, look at some of the topics we put out. We don't repeat things a lot, but we do have the same technology areas. So you can get a feel by looking at the stuff we put out before, kind of the areas that we've been looking in, and then reach out to the program manager. So for me, we have on our website, we have links to all of the 13 DOD component webpages. Reach out to those program managers and they can help link you up with the right people over.

Terrific. Stephanie, anything you want to add there?

No, I mean, I would add that, and I think this is True across the agencies is reaching out is a great first step for us at HHS. You can go again to seed.nih.gov if you look for the picture on the website. Scroll down to the right. If you look for the picture, there's a big picture of the NIH campus. Looks very regal. But there is a list of program contacts for each of the 24 different institutes and centers at the NIH, as well as for CDC, FDA and ACL. Oftentimes, just looking at the specific missions of each of those institutes and centers can be enough to say, hey, that's where I need to go. But again, if you're not sure we are here to help, please reach out and talk to us and we can help guide you and connect you to the right individual. The last suggestion I would make is that our omnibus solicitations do have program descriptions for each of those different 24 institutes and centers. There's a wealth of information there and I encourage you to read that document that also might help you determine what might be the best fit.

Great.

And Ben, what's the situation at nsf?

Yeah, I would endorse everything that's been said. I guess three very quick tactical recommendations for NSF specifically. Number one is that every Thursday afternoon we have open office hours for applicants where you can ask questions and have a program director answer them or just listen to everyone else's questions. We get through 30 or 40 questions that give you a flavor of what's relevant and what the key issues around the program are. Number two is that if you're a person who thinks in kind of context or analogies. We have actually a pretty good keyword searchable portfolio page on our website seedfund.nsf.gov you can put whatever technical business market keywords in and see which of the 3,200 companies that we funded over the last decade are in the same area. Get a sense of what their projects are and, you know, just kind of get a feel for the types of things that we support. Last thing, which is the most important one, which is that we actually have a formal process for establishing appropriateness called the Project Pitch. Nancy, you were instrumental in helping start that when you were at nsf. It is something that is required. It is a lightweight two page summary of what you think might be a project that NSF would be interested in funding. It goes to a program director who is the final decision maker and they will give you a sense whether it might be a good fit or that it won't be. And either way, you'll get feedback and if you're lucky, you'll get an invitation and more information about how to submit. And you'll have the person who is your program director responding to you so you'll know who your point of contact is as well. And so that project pitch gives you a year, an invitation that's good for a year. So it's something that you don't need to wait in time. Perfectly. It's something you can do even before you've actually incorporated your company. If need be.

Great. And as a follow up, I know that I can go to Ben mentioned a website or I can go to sbir.gov and under reports and awards I can search for projects or companies that have been funded in the past. Okay, that was my plug for going to sbir.gov but anyway, if an agency has funded similar companies or technologies in the past, is that mostly a good thing or a bad thing? For me, if I'm an entrepreneur and hoping to get funded, Ben.

I'm not sure that's a big factor either way for us. You know, 3,000 companies and some of them are going to be doing similar things. That's kind of inevitable. I think it's important that everybody understands their competition and the landscape and status quo of the market they're going into. Nsf, we're very focused on market risk, right? Market risk is something that our funding doesn't directly tend to address, but it is the biggest risk that kills our companies. And so whether or not NSF has funded it, if there's another company operating in the same space that is a direct competitor or that is relevant to your business model, it's important that you convey that your understanding of the existence of that company and what it tells you about your, your own path to market as well. But it's less important whether NSF specifically has supported that company. We don't, we don't shut the door behind a given company in a given area once we funded them.

Stephanie, anything to add?

Yeah, I think that's, that's absolutely great, Ben. And I think that's a great way to think about it is really understanding your competition. But again, we're not shutting the door just because we funded. We're trying to really look for solutions to a problem. I would also point out that while sbir.gov is great to really understand what has been supported in the SBIR landscape, at NIH, we have reporter.nih.gov so reporter.nih.gov, which provides information about all of the projects that NIH has supported, not just in the SBIR space, but more broadly. And again, sometimes it's not that someone you know, we've funded similar technologies or solutions to a similar problem. But that helps you understand the overarching landscape and it enables you to write a better proposal that takes into consideration that that work is already out there. And you're able to describe to our reviewers why your work is different, what you're going to add to that landscape.

Susan, anything to add?

Yeah, I'd like to add a couple things to both of their comments. So first of all, yes, absolutely. Know your competition, know your, Know the landscape, state of the art, but also know your customer. DoD the department is the customer and you have to know us as well. So reviewing our websites, looking at what we funded is very important to get to know to get to know the customer. Also, I would say that the department funds many multiple award winners. So for example, we are going to fund more than one phase one per topic based on the quality of the proposals we get in. We are going to fund probably at least one or two phase twos per topic. So it's not necessarily about the number of proposals that you have, but the quality and what you're offering us for the solutions. So I would say that those are great things to think of the market and also who might you be able to partner with? Because sometimes if you're trying to put together something that's a component of a larger platform or system, your technology plus somebody else's might work together better than the two of you by yourself. And that's always a business decision. But it's also something that might make you more look better to us than just by yourself. So those are some things to consider when you're submitting proposals, I think, to anyone. And we definitely know that sometimes it takes more than just one phase one or two phase twos to get a technology to a point of maturity where the department is interested in actually being your customer. So sometimes you have to be in the game for a few cycles for that to happen over.

Just drilling down on the importance of customers and commercialization. So I think you've all said that it's important to understand the customer in the market. But as an applicant or an offeror, how do I show that I understand the customer or the market, especially in a phase one application? How does an applicant convey that in their writing? So, Stephanie, can you start that question?

Well, as noted, the NIH and HHS in general isn't the final purchaser of the technology Those customers are going to be patients, clinicians, caregivers and researchers across the United States and potentially the world. Within a phase one, the HHS doesn't have a separate commercialization plan in the phase one. So the phase two has a commercialization plan and you can put that information and detail that out there. But in the phase one, it's really important to focus on the significance and innovation sections. So the significance is where you talk about the significance of your project, the problem, and how this is going to solve that problem. Innovation is a great place to put how you are different from that competition. What is the innovation here? And again, innovation doesn't have to be about innovative, the innovative processes. So I like to joke the, in general, getting a regulatory going through the regulatory process is not an innovative process. There's a standard process to that. But the product itself or the service itself is going to be there. So that's where you can really put that information about how you understand the problem and the customer and then how you're going to address them.

Susan, you want to take a stab at that?

Sure. And I think it, I think you also have to kind of watch your page limit too, right? So you have to kind of make sure that you get that across within, you know, not take up too much of your proposal. And it's, it's, it's kind of hard to tell people this, right, because you want to make sure you get that point across, but you don't want to take half your proposal to do it. So you might need to maybe weave that into the elements of your proposal, depending on the number of pages that you have. You know, weave that into it so that you show your understanding, but also show how what you're doing is going to answer that or exceed our expectations for what we've asked for over.

And Ben, what do you have to add?

This is a good contrast of that as that adage that every agency is different. So you are going to spend half your proposal that NSF telling us about the market. That's actually the recommendation of the split between technical and commercial content. And because the companies we fund are again, very early, even if we ask for a full commercialization plan, you probably don't have one at this stage. We're going to fund. The average company we fund has two employees and is one or two years old. That's the standard phase one awardee. So it's not about a commercialization plan because you don't typically know enough to have a plan, but in the sense that a plan is A is a way to go from point A to point B. What you should do in phase one at NSF is convince us that there's a point B. Right. And that essentially boils down to two of the most important sections of your business model, which are the customer and the value proposition. This is the reason why NSF created the I Corps program, which is now all over the government, which is great. It does, it does teach customer discovery, which is the best known method for evaluating customer and value proposition at the early stages. It's why we teach every phase one awardee through the bootcamp that we offer for free. Customer discovery. And if you, if your company wants to show that they've been thinking commercially before phase one, they should have engaged with customers meaningfully and repeatedly and again, not just to talk about their technology. So the important thing for folks to remember if they, they're not familiar with the practice of customer discovery, you should, you should learn about it. If you're starting a company, customer discovery is not about the car you're going to design. It's about how much your customer likes their horse. Right. There's a difference. And it's important to keep in mind that a letter from a customer saying that the technology seems great is about the least useful thing for NSF and I probably for other agencies because we have technical experts who will tell us what they think about the technology. That's not what we need from you in your proposal. What we need to know is what your customers think because we don't have your customers in the room. Susan does, probably a DOD I think Stephanie and myself, we don't have any, any direct customers of most of our applicants in the process of review. And so you need to bring that voice of those people into the proposals that they're. So that is present.

Super helpful. Okay, so let's get down to specific. The time from application to funding. So how long does it take from the time I submit an application or proposal and maybe that would be at the deadline to actually getting funded, if I'm so lucky to be funded, and will I hear anything in between the time of submission and yay or a nay. Stephanie.

So in general, and there is some variation here, but in general, if you're talking about from the time of application, that deadline to actually receiving funding, that can be six to nine months. However, you will hear about three to four months in because we have a two step peer review process. So you'll hear about how well that first stage of peer review went and you're going to Hear that? The three to four month mark. So you're going to get feedback and an indication as to whether or not you're likely to be considered for funding at the three to four month mark. So while six to, you know, six to nine months sounds very long, you actually get feedback and input back much earlier.

Great.

Ben, how is it the same or different at NSO?

The timeline is probably similar. 6 months to award, maybe 7 months. In some cases you will also hear, in some cases you'll hear something a little sooner than that. If you're competitive, if you're still on the running, if you're not competitive, you're not near the kind of fundable range, you may just hear a no. Either way you're going to hear all the written feedback. We do bring in external experts, as I mentioned, all their written feedback without any edits will be sent back to you. It can be actually pretty useful feedback for new companies. At least I benefited from it and I think a lot of our PIs tell us that it's very important. But yeah, most of the process either way you won't hear anything if you are competitive. At some point people start to ask very pointed questions, which is a good sign but not a guarantee for our process. And the program director obviously has their own sense as well. So one piece of advice I would give folks is that if you don't get funded and you do get that feedback back from our peer reviewers who are anonymous, you also want to probably reach out to the program director and see if they have anything else that they had on their mind that they'd like to share because that is not necessarily conveyed as obviously and as directly as the external review feedback.

Thanks and Susan, okay, so like the others, it's going to vary. Phase one you're going to see from solicitation close to time of award, you're going to see much quicker than a phase two. We have to. For both phase one and phase two, we have to provide a notification to all proposers within 90 days of solicitation close. And if we can't do that, we have to ask for a waiver from sba. So you will receive that notification at least within the 90 day period whether you were yes, selected, no, not selected. And again, phase ones will be fairly quick award because those are shorter timeframe and they're usually firm fixed price contract while phase IIs are typically cost plus type contracts which take a little bit longer. And so those could take anywhere from 90 days to six months. And sometimes, and I hate to say this, but sometimes longer, depending on kind of similar to nih, we do have human animal testing that's required. And so sometimes getting those review board approvals take a long time. So we typically recommend that you don't propose or. Let me see, let me say that again, that you, that you separate any kind of testing from any kind of research because then it's easy for the contracting officer to separate the award so that while you're waiting for the approvals to be done, the other stuff, the research part can be awarded. And then when you get the approvals for the testing, whether it be human or animal testing, you can get that that could be added to the contract that's been awarded. So there's quite a few variables that can come into play if you, especially for phase twos, feasibility studies, you know, that's pretty much a paper deliverable. So that's not as, that's not as complicated, but that's our timeline over.

Great. So let's move on to the sad situation if I didn't win funding on my first try and talk about reapplying. So how important is it at your agency to reapply if I don't achieve an award on the first try? Or another way of asking the question is how often do the people who, the companies who actually get the awards, how often are they actually getting it.

On the first try?

So Stephanie, could you start that one?

Absolutely. So resubmission, which is another way of saying reapplication. But resubmission is very common at nih. Very common. The majority of our applicants will have to submit more than once. That is just the fact. Now, like Ben suggested, we also will give feedback form that peer review. Everyone, regardless of how well you scored or how well that peer review went, you're going to receive feedback. We always encourage people to read that very carefully and then reach out to the program officer who's listed on that feedback. Those reviewer comments or summary statement talk to that program officer. As a program officer and I was a program officer for many years, I read a ton of summary statements. I got very good at may be your first summary statement, but it certainly isn't mine. And so we can really help you as you're trying to determine what are the next best steps for this project. But it's very, very common to resubmit. So I would say another thing to keep in mind here at HHS and particularly at NIH, about 40% of our companies are new to the SBIR STTR program. So for our new projects, about 40% companies who have never received an SBIR or STTR before. About half are new to the nih. So a little bit, a couple of companies maybe have done an sbir, sttr. But it's so important to reach out and talk to us once you've received that feedback. I'm going to do one more quick plug. The peer review process, particularly for new individuals, can be a little daunting. It's a little like a black box. The best way to learn about peer review is to be part of peer review. We are looking for peer reviewers, so you can also contact us@seedinfoih.gov and request to be part of the review process.

So.

And nothing will make you learn that process faster than being a peer reviewer.

Great tip. Ben, what would you like to add?

Yeah, so the numbers for NSF for first time applicants. So folks who are submitting their first ever proposal at NSF and get funded is about 60%. So it's actually a majority of our phase one awards where the first ever grant NSF got from a small business. The number of folks we fund who have no prior SBIR funding from Any agency is 92% roughly. It fluctuates a little bit, but it's around there. So it is almost all, maybe some repeat applicants, but almost all first time award winners. So I think as far as reapplying, it's maybe not quite as important because I think we weigh again our focus on trying to fund early is very important and so we'd rather find an application that maybe has a few warts but is early enough for us then maybe fix those things and have them come back later when the stage will be not as appropriate for us. Right. The companies that we fund in general, we're looking for them to move quickly and so you know, for the greatest companies they don't stay start for very long and you know, once they're out of that phase they're typically we can't help them anymore. So reapplying is fine, but I think it's less of a requirement for NSF than maybe for other agencies. You know, in the goal for working, of working with startups and supporting startups, you have to basically relax yourself a little bit on the grantsmanship. You can't assume startups are going to be able to do that. That's not a natural tool of entrepreneurs. And so you know, we've come to peace with that and trying to be forgiving and flexible with people making little mistakes and administrative errors on their application.

Great. And Susan, what would you like to add.

I would also say that reapplying so I guess I need a clarification on what reapplying means. So if you're saying reapply to the same because I think that some of the other agencies have broader announcements that are open and you're just reapplying to that same thing. Well, we don't, we don't work that way. So each announcement is its own separate announcement. So that topic, it will be rare for you to see that exact same topic again unless we didn't get enough quality proposals to that topic. And then we probably won't know that for a year or so. So you probably, if you do see that topic, it's probably not going to be for another year or two. But we don't run the same exact thing over and over and over again. And then I would also say that it's important for companies that have had an award, whether it be SbirsttGR phase one, phase two, it's important for you to tell people that you've had an award because the department will we call out, they're called, they're formally called cross agency awards now. But the department we used to adopt things before this was formalized in the statute, we would adopt things from other agencies. And so if you've had a phase one or a first, phase two with another agency and we're interested in it, then someone on our team will probably reach out to the agency first. So if we're interested in something NIH or NSF started, we will reach out to that organization first to make sure that we can take over that topic that they started or make an award against it. And then we'll reach out to the company and ask for a proposal. So it's very important to keep your data up to date on sbir.gov and then also market your SBIRS and your STTRS to people and let them know because not only do we leverage each other's investments that we've made through the program, but when it comes to phase three, which is transition or commercialization of your technologies, you have there is potential for a direct award be made to you without further competition. So it's important that you make sure people understand that that is a benefit to this program and that you have had an award or multiple awards under the programs over.

Great, really interesting information. So now we're going to move into the part we call the lightning round. We'll see how lightning we are. So I'll start with the first question, which is should my application be broad, like should I be describing a platform or should it be specific, like on a specific use case? So, Susan, you want to start that.

So respond to the topic. The topic is going to pretty much, if it's, if it's specific, it will have the specifications that they want you to answer in the topic. And if it's not, it'll tell you that you can propose multiple different ways to address something. They will not be specific. So when you read the topic, it will tell you that over.

Great.

Ben, we fund a lot of platforms and they have a lot of use cases. And so we want you to be broad on the mission of the company. We want you to be specific about the near term beachhead use case and market because typically you want to do one market at a time. Start somewhere that's, you know, maximally advantageous to you and then expand from there. So be tactical and specific about the market and be broad about the mission, the technology.

And Stephanie, it depends. For our specific notices of funding opportunity or the specific topics, when we put out a contract solicitation, you are going to be addressing what is in the notice of funding opportunity. For the omnibus solicitation. We do fund some platforms and some of our institutes and centers are really focused on developing those platforms. But for many of our institutes and centers, we're interested in you coming with a solution to a specific topic.

Okay, great. How about this? Is there a roadmap or a checklist that I can use to develop a good submission to your agency?

Stephanie, there is a wealth of information on our website. I'm going to plug our website again. And this is why we actually have application resources on our website. We have an annotated form set, we have sample applications, we have instructions. We really are trying to make it as easy as possible with that application process. Finally, if you've not received an nih, SBIR or STTR before, we do have applicant assistance programs that are available. So there is a number of things we can do. I will say the one thing I can not really help with is the registrations. So please, please, please start those registrations now. If you don't have them, there's nothing I can do if there's an issue with the registration. So please start those registrations now. It hurts at least one or two companies every receipt date and there's nothing I can do about it. So register early.

Okay, Susan?

Yeah, I will foot stomp that one too. You have to have a unique entity identifier number. The former. The former. What was it? Called Duns and Bradstreet number. I know that you have to register an sba. You have to register. You have to have an account in our system. And we try to make all of this kind of seamless, but it isn't always right. I mean, it's. It's different systems. We try to communicate with those systems, but sometimes there are glitches and if you wait till the last minute, it could really mess up your getting in or getting your submission in on time. Because we do not accept late proposals at all, period. Unless there's a. If there, if it's a problem on our end, then we have to do the, you know, we have to do an investigation to make sure it was something that was on. On our side. And then that point we will. We will reopen or we will extend the solicitation so that everyone can submit. But that's a rare. That's very rare.

But what about a roadmap or a checklist? Like how to put together a good submission checklist?

Basically, our solicitation instructions tell you everything you need to have in your proposal. And then our proposal submission system is very helpful because it will flag you. It's not going to a lot of things that will not stop you from submitting a proposal, but it's going to give you a warning. So if one DoD component wants 20 pages and you've got 50 pages, it's going to be like, hang on, you've exceeded the page limit because each component can configure their submissions to suit their needs. So it's not going to stop you, but it will warn you. And then there's a few things that it will stop you, like if you haven't met the required percentage of work requirements or something like that, it will warn you that it's not correct, and then it'll stop you from going any further. So you're going to have to fix that. But other than that, reading the solicitation carefully, it tells you exactly what you have to have in there. Over.

Great. And Ben?

Yeah. So nothing much to add, except for that section 5 of our solicitation is the closest thing to a checklist of the pieces of a proposal, and that we have a very comprehensive faq. Those would be the two documents I would refer you to.

Great.

Okay.

So after the thousands of applications that you've read, what are some of the ways that applicants can set themselves apart? I'm asking, like, what are the success characteristics of a successful application? So, Ben, can you start this one?

Yeah. So I think for nsf, I think the most important things are number one, as we discussed, you have done consistent work on understanding your market. Usually that's also a proxy for what you do when nobody's paying you to do anything. And that's a very good indicator because you're going to have to do a lot of unpaid market de risking because we're not paying for that. Right. So you showing that you can de risk the market without support is the most effective indicator I have of your commercial success. The other thing is that the company needs to be structured in a way that I see ideally at least one person who's going to wake up in the morning trying to make this company successful and who has cleared their calendar for the next five to 12 years. Right. That this is at least one person's primary professional priority. I know you'll have a lot of part time help and advisors, but in my experience the companies that we support cannot succeed with all part time employees, even at the very earliest stages. There has to be one person who this is the most important thing happening professionally for them, ideally with a lot of support or maybe two co founders, even better. But if nobody is willing to actually drop their other commitments and do this thing, this startup, then no one is going to make it happen in my experience.

Thanks, Susan.

It's really, again I just go back to, it's really important to read the topic and respond to that topic. You know, we've, I've seen topics or proposals where the company submits an idea that, that isn't what we've asked for. And so that's totally non responsive. You know, if I've asked for a boat, don't give me a plane, you know, that's kind of, that kind of thing. So really responding to the topic, following the instructions, you know, looking at the evaluation criteria, you know, we, they're usually listed in descending order of importance which the technical merit is number one, the key personnel and the personnel to commercialize. Kind of like Ben said, somebody that's going to be, that's experienced and that is going to be, you know, there with the, with the, to shepherd this project through and then your, the past commercialization experience. And if you don't have a lot of commercialization experience, that doesn't, that doesn't necessarily work against you depending on the type of technology it is because as I said earlier, we want all stages of risk. Right. We want innovative, we want disruptive, we want things that have never been done before, that kind of stuff. So if your company's never had any commercialization or transition experience, it doesn't necessarily hurt you, but you do have to demonstrate why you're in your proposal by the three criteria. You know, what your, what you can do. And then, and then aside from that, I would say to be very careful about doing things after you've submitted your proposal, like changing your name of your company, changing your firm, your firm, the main person in your firm, or your PI changing anything that you submitted in that proposal. We've had this happen many times where the company decides to change their name and then it totally messes up and slows down the award process. If we can even award to you at all at that point. And then swapping out people, you know, we're evaluating you based on the, on the people, the team and the work that you submitted, not some substitution afterwards. So that can really put your proposal at risk if you don't have solid commitments from the people that you've proposed on your team and that sort of thing over.

Stephanie, Sorry Nancy, can. I don't want to, I'm going to screw up your lighting right now. But I think this is a really great point of contrast at NSF. We have 42 companies recently that we've identified that we funded that have become quarter billion dollar or higher valuations. 36 of those companies had no co founder with any entrepreneurial experience. And so I just want to put a contrast that almost all of our teams have no experience and actually in some sense that can be a good thing. It's not necessarily a good thing, but in our experience when somebody says your company needs business experience, we don't necessarily weigh that as heavily, I would say, because we know that many startups that you and I are familiar with were led by people. Pick your favorite startup metaphor and tell me whether those folks were first timers or not. So first timers are great if they're committed to. And at nsf we're absolutely not looking for people who are serial entrepreneurs necessarily.

Well, part of why the audience is so lucky today is that they're getting to see really that's what this panel was about to see the differences between the agencies. And it's really coming out. So I think the audience will really appreciate that.

And I think we're more like NSF in that regard. Many of our companies, this is the first time they're coming. These are individuals who have never done businesses before. They're first time entrepreneurs. Many of them are trained researchers. They may have a real grasp and understanding of the technology, but this is their first entrance into the business world. And that's okay. We've got a number of resources within my office to specifically help support those individuals when they become awardees. We have a wealth of resources for them. To your question of what makes a successful, you know, specific application, what really has someone stand out? And I'm going to make the assumption that they've read the notice of funding opportunity very carefully and reached out and talked to us. Because those are two basic, those are basic points. I think it's really making sure that you have the right team of people around you and it doesn't necessarily mean that they're on staff, but you've thought about the problem and you've really talked to experts in the field. And I say this because we found many situations where somebody creates a solution to a problem and they may have the expertise in engineering, but they may not have the specific disease expertise of the disease that they're going to solve the problem for. And that disease expertise can be very important. So it's really understanding what the customer will want, understanding the problem and understanding the landscape that you're in. What has been tried before, what has been successful, what hasn't been so successful. So I think that's, it's really being able to understand that and then present that in the application that's also important. Although I think Ben made some great points earlier and I think those are just as important as well.

Great. The next question has to do with the decision making process at your agency. So who actually touches the application once it comes in and who or what, what being a committee maybe, but who or what actually decides who ends up getting the funding? Susan, can you start on that one? Sure.

So within the department that's going to go, that's going to be decided within each of the components that put the topic out. So it will be a group of people within that organization, a contracting officer, tpoc, some other experts, if the TPOC needs experience with evaluations, assistance and that kind of thing. So it's going to be, it's going to depend between each of the organizations who is on that team to make that final decision. But there is a decision authority that is usually usually one or two levels above the person that wrote the topic. And then.

Those are all people that are internal?

Yes. We do not do peer reviews. The department does not do peer review. It is all internal government people.

That.

Understand what the department needs, what the mission, how it's going to respond to the mission, needs that we put out in the topic. And so it's going to be people that are familiar with that and the desired outcomes for the platforms or the programs that we're trying to put this technology into. So it's going to be those folks that are involved with the decision making process over.

Okay, how about at hhs, Stephanie?

So at NIH we do have two levels of peer review. We have the peer review panel and then the second level of review is with the advisory council, which is also made up of individuals, both researchers, clinicians as well as individuals from the community. But all of these are advisory. I think one of the myths that I often hear is that peer review makes the funding decisions. Peer review doesn't make the funding decisions. There's due diligence that's done internal to NIH by program officers and the final individual who makes the final funding decision at an institute or center is the institute or center director. So all of that is advisory. It's very important that you listen and respond to peer review. But NIH staff takes that information, presents that to the institute directors who make the final decision.

And how about nsf, Ben?

Yeah, we do bring in external experts. They give advice and they give you feedback. Like Stephanie said, they are advisory and the program director in our case makes the final decision. So myself or my colleagues, depending on the portfolio area and at nsf, I think in the small business programs, the correlation between the final recommendation by the program director of what to fund and what say the peer review group would have funded. Funded is actually not as high as I believe it is. In some other areas I would say it's something like 70%. So yeah, a lot of it, a lot of the times we do agree with the reviewers, but sometimes we, we have our ability to go a different way.

Okay, my last lightning round question is a kind of negative advice, I guess. What's the one thing you would tell an applicant? Don't do this. Like definitely don't do this. So I'll start with Ben.

How do I narrow it down to one? Let me. I'm going to break the rules again. Two quick things. Number one is don't say there's no competition for your product. Competition is whatever the market is doing now at least, and probably competition beyond what you, a narrow focus would consider as well. But there's always competition. Second thing I would say is just around structure. If you want to own the company, you want to control the company and you want to bring a new technology to market, go full time with the company. And if you're not going to be willing to go Full time with the company, hand up, hand it off, hand the reins, hand the equity to somebody else who's willing to do that. Otherwise you're just squandering the opportunity.

Susan, what would you say? Don't do this.

Don't wait till the last minute to submit your proposal. That would be the main thing, I would say. I would also say don't waste your opportunity to use the pre release period to ask technical questions and don't be afraid to ask for help. There are the department has, they were formerly called procurement technical assistance centers. PTACs. Their name is being changed to Apex Accelerators. Those folks can help you figure things out. So don't get frustrated and not ask for help. If you need help, there's plenty of resources to help you out. And again, just foot stomping. Don't wait till the last minute to submit your proposal. And I'll just tell you we've had people logged into the system and then 12 o' clock rolls around and they get kicked out and they think there's something wrong with you, the system. It's like, nope, you. Just because you were in the system while it was, while it was still open doesn't mean you're done, right? So if they haven't hit that final submit by noon and they're still in the system, it doesn't matter. It's going to cut them off. So please do not wait till the last minute. You've used up all your resources. Don't do that. That's my number one.

And Stephanie, don't wait till the last minute. Don't wait to contact us. Don't avoid asking for help.

Okay, great. And now we have the final lightning round, which is your chance to say any other piece of advice you want to give to the people out in the audience or if you want to ask yourself a question and answer it, that's fine too. So let's start with Susan.

Okay, so first thing I want to do is clarify that. If I made it sound like we're not interested in new companies, that wasn't my intent. My intent was just to make it clear that we want those things in the criteria. It's not going to be, it's not going to be held against you if you don't have commercialization experience. But we want to know if you do. And please, please play that up, you know, give us everything that you've got, whether it's with another agency or whatever. So I just want to make that clear and I want to say that the nation needs you. We have national security challenges that need innovation and we need your innovative technologies for our war fighters to give us the technological edge on the battlefield now and for the future. So please consider responding to the Department of Defense. Thank you. Thanks.

And Stephanie, we have like 30 seconds.

Offer to be a peer reviewer, contact us and have other people read your proposal. That is one of the other things that you can do. Have others read your proposal, ask for help.

That's great.

And Ben, what Stephanie said, and let me just finally say there are four things that I've been told you're never ready to do. Get married, buy a house, have a kid, and start a company. If you think you're not ready, just start it anyway. It's an incredibly important thing to do. It's incredibly hard, but incredibly important for our country, as Susan mentioned. So best of luck to all of you.

Thanks. And thank you to Susan, Stephanie and Ben for joining us today. And a special thank you to all of you, our audience, for being here and learning more about America's Steve Fund. Please share your thoughts about this session and the other sessions from today via the link that we're sharing in the chat. And we also encourage you to browse the agency resources in the exhibition hall. So I hope to see you back here tomorrow for the agency open houses. Beginning at 2pm Eastern, each participating agency will host its own breakout room where you can join and then ask your questions directly to agency representatives. So thanks everybody and see you tomorrow.

Sa.