LLMs by themselves don’t learn from past past mistakes, but you could cycle inference steps and fine tuning/retraining steps.

Also, you can store failed attempts and lessons learned in context.