I’ll be the first to confirm that life is not fair, things never go the way you hope, nor are all opportunities seized upon. That is all to say there are always plenty of reasons anyone to complain, and as such a constant drum beat of many to their dissatisfaction of the current situation as they see it. But that doesn’t mean that hope is lost, and it obscures the many whole toil behind the scenes to build a better future.

I had the opportunity the other day to have a fascinating discussion about roadblocks to change in the DoD, especially as it applies to Great Power Competition. One of the main points brought up was the philosophical differences between service providers and warfighters, and how that fundamentally influences all resulting aspects of their operations.

Every person in the Air Force has a series of interlinked personnel based activity reports that they are either required or optionally available to them relative to performance. These reports have an interlinked position in time and scope, each building from the previous, and supporting the next in the chain. A simple example would be personnel performance. At the end of the chain you could have a medal package for the award in the recognition of achievement, and you could follow that record of action that justified the achievement back through a series of documents each decreasing in scope and time, all the way down to the smallest and shortest form for weekly activity tracking.

With so many people making the pilgrimage to the DC area for AFA this week, I thought this would be a timely topic to bring up. Remember that LLMs fundamentally rely on mass amounts of data to function well. As good as scraping the entirety of the internet is, I think we can all agree that’s neither authoritative nor necessarily as deep or accurate as any topic actually goes. So where are the most detailed and accurate sources of knowledge on any particular domain? In the minds of the people that work in those domains everyday.

Unlike my four previous posts on the topic of our AI study on LLM technology, this one is more a collage of items that didn’t really fit well enough to include in the written presentation. They might come up in a verbal presentation, but not likely unless a question drove the interest in a deep dive. But I like having some sort of written record beyond just my endless pages of random notes, so I thought this might make an interesting post to share.

Continuing on in the series of lessons learned from the chatbot study, let’s look now into the art of prompt engineering.

Continuing on in the series of lessons learned from the chatbot study, next let’s discuss use case discovery.

Continuing on in the series of lessons learned from the chatbot study, we find a clear but important distinction.

It is time to conclude the AI study focusing on LLM-based Chatbots I have been leading as my current MPA orders conclude with AETC. As I have no expectation that I will get approval for a full public release of the study results in either PowerPoint or Research Paper formats, I wanted to make a series of posts focusing on the major thematic lessons learned instead. All of these are completely unclassified, and won’t include any sensitive use case data or direct descriptions. Ultimately though the real value is in the knowledge gained not the specific assessments conducted.

Made famous by the remarks of Donald Rumsfeld in 2002, the quad chart of below can provide valuable insight to the application of LLMs within any domain. There are two continual issues with LLM utilization that both functionally and technically are unavoidable.

Plenty of companies out there pitching custom trained Large Language Models (LLMs), and/or their services to custom train LLMs for the Air Force on our data. To save everyone’s time here’s the absolutely free and publicly accessible crown jewels of the Air Force’s information kingdom. Let the best model win!

Though I know this might be upsetting to some. The chart above shows some general terms of how the decision and action matrix works for acquisition decisions across a range of variables, but this is mainly depicting cost as the central organizing metric.

Military Innovation Professionals have to be skilled in three interrelated areas of Acquisition: Program Management, Funding, and Contracting. Let's play some Innovation Moneyball with tricks of the trade.

Concepts, roadmaps, and pipelines a plenty have been thought up to solve the innovation-to-acquisition problem. In simplest terms how do you get a product or project from start-to-sustainment? Let’s skip over all the wishful thinking about what might happen in a future DoD Acquisition reform, and instead help create capabilities today in the system we have.

The innovation ecosystem can seem to be a chaotic fog with many players stumbling blindly trying to find their way to seemingly lost finish line. What is most needed is teamwork to align common needs with common goals and pooled resources. But how do you bring participants together when you might not have the usual tools of a military hierarchy to fall back on to force a cohesive game plan? It all relies on one magic word...incentives!

In the final installment in of my three part article series providing a more in depth look at the Innovation acquisition system, let us now turn to the most mysterious and hard to answer question for outsiders of “who do I talk to?” To answer this question it is first key to understand the organizations involved and their primary function either in or related to acquisition. The title graphic above shows the layout of major organizations that have required roles in this overall process. The ever elusive needs, requirements, and funding decisions are all made at the Combatant Command (COCOM) and Major Command (MAJCOM) level. The outcomes of those decisions are then acted upon for R&D or Acquisition by internal or external organizations depending on the size of the effort, with large efforts going external and small ones staying internal to the commands. Every command staff serving both COCOMs and MAJCOMs have statutory functions they must provide, and therefore have similar but varying structures and even these change over time. Understanding these structures is the key to figuring out who to talk to for any major decision in the acquisition process.

Executing a successful innovation project stems from the combination of two fundamental skills across the lifespan of the effort. First, a solid organization plan implements everything from the pitch, through the work plan, and ending in the test results or project report. It is doing all the actions required to not just heard the cats, but properly express those plans in written form for others to see and understand. Second, the ability to tell the story of the project to each applicable audience in the right way at the right time is the lynch pin to gaining and carrying forward support to the project over time. These two skills are what holistically grows, supports, and maintains a project over its full lifespan.

Many times in the process of military innovation, we find ourselves in organizations with only one significant resource. Often thoughts immediately run to how do we build up the trifecta of resources to then start doing things ourselves, versus trying to partner with others to start things today. There’s always an opportunity for creative partnerships between organizations that only requires creative planning and open minded participants. Let’s look at some examples of each situation.