I have been working with AI since the ‘90s, case-based reasoning since 2010, and generative AI for the past year. I have seen it go from a wonder in the eyes of the more geeky public to an eyesore as they put in terrible prompts and then berated the answers.
I have heard of lawyers using hallucinated case law to justify their position in court. And perhaps most egregious: I read there was Chicago pizza in Florida. When I asked chatGPT it told me the name Nancy’s Pizza, with an address only 30 minutes from home. How could I not know: Nancy’s Pizza, who I give credit for inventing Chicago Pizza, actually had a location nearby. As I calmed down and started planning a trip, I checked Google Maps. No Nancy’s pizza in Florida. Oh no… How could I not have supported Chicago Pizza, and now it was gone? Using old-fashioned Google, I learned there had never been Nancy’s pizza in Florida. Just a Hallucination. This was probably 6 months ago, before chatGPT 4o. I just tried and ghatGPT only recommended two real pizza places in the area.
Now I describe AI’s timeline as “one AI month is equal to 8 dog years” So now I ask chatGPT (or the other AI models I am leaning toward) to justify their answers and warn me of hallucinations.
I have been spending significant time trying to identify where generative AI goes wrong and how to remove hallucinations, first with prompt engineering, then with fine-tuning, and RAG (using our own data or third party data) lowering the temperature (reducing creativity of the AI model) and more.
I am fortunate that I am working with AI from several directions: Non-Profit organizations providing answers to sometimes hard questions, and SEERai, describing the characteristics of engineering products so the SEER models can provide cost, schedule, and risk.
I realize anything I write may be wrong in a few months. For example, just this weekend, a new AI model based on Llama was introduced that allegedly self-corrects errors. And now people are contesting it, illustrating my dog year analogy.
I will be blogging about this journey and the lessons learned.
Navigating Estimation Bias Across Software Development
In today’s software development landscape of ever-changing requirements and uncertainties, it can be difficult to manage projects, ensure each initiative is rooted in data-driven insights, and stay on track for success. However, it doesn’t have to be a time-consuming and demanding process….
Unlocking the Power of Historical Data for Accurate Bid Cost Estimates
By Matt McDonald, President, Galorath Federal Inc. Businesses—whether service providers or manufacturers—are constantly under pressure in today’s world. They need to efficiently bid, manage, and monitor costs, schedules, and risks. Many companies are focused…
How Cost Estimation and Project Management Work Together for Success
Managing a project successfully means juggling several crucial elements, like scope, deadlines, and budget. While each plays a vital role, having an accurate cost estimate is essential for resource allocation, preventing overspending, and ensuring…
ICEAA/SCAF Symposium Recap: A New Era for Data Interaction Through Generative AI
The recent 2024 ICEAA/SCAF International Training Symposium in London highlighted the growing impact of artificial intelligence in cost estimation. Galorath demonstrated its SEERai® generative AI solution, showcasing its ability to reshape the cost estimation…
Maximizing Price-to-Win Success with SEER and SEERai: A Comprehensive Approach for Analysts and Stakeholders
By Chris Hutchings, VP Global Solutions, Galorath Incorporated In today’s highly competitive market, Price-to-Win (PTW) analysts play a pivotal role in determining the optimal pricing strategy for complex projects, particularly in defense, aerospace, and…
Share this:
Get access to news
Sign up to get relevant news in your inbox - never miss it!
