[Diff since v3.19.2](SciML/Optimization.jl@v3.19.2...v3.19.3)

**Merged pull requests:**
- Add OptimizationPRIMA to the docs (SciML#616) (@ChrisRackauckas)

[Diff since v3.19.1](SciML/Optimization.jl@v3.19.1...v3.19.2)

**Merged pull requests:**
- CompatHelper: bump compat for Optimisers to 0.3 for package OptimizationOptimisers, (keep existing compat) (SciML#580) (@github-actions[bot])
- Documenter 1.0 upgrade (SciML#598) (@ArnoStrouwen)
- Improve Performance for OptimizationBBO (SciML#600) (@Zentrik)
- Add callback to MOI (SciML#601) (@Vaibhavdixit02)
- Mark tests broken on v1.6 as broken on v1.6 (SciML#603) (@ChrisRackauckas)
- Bound Optim.jl to the range where it is generic (SciML#604) (@ChrisRackauckas)
- Require v1.9 for AD tests (SciML#605) (@ChrisRackauckas)
- build: lift Optim bound (SciML#607) (@sathvikbhagavan)
- build(OptimizationOptimJL): bump patch version (SciML#608) (@sathvikbhagavan)
- Make AutoZygote robust to zero gradients (SciML#609) (@ChrisRackauckas)
- fix typos (SciML#611) (@spaette)
- [WIP] Add PRIMA wrapper (SciML#612) (@Vaibhavdixit02)
- Fix zygote constraint bug and update rosenbrock doc (SciML#615) (@Vaibhavdixit02)

**Closed issues:**
- Inclusion of PRIMA solvers in Optimization.jl (SciML#593)

[Diff since v3.19.0](SciML/Optimization.jl@v3.19.0...v3.19.1)

**Merged pull requests:**
- map `maxiters` to `outer_iterations` (SciML#544) (@SebastianM-C)
- Eliminate some runtime dispatch and other things (SciML#597) (@Vaibhavdixit02)

**Closed issues:**
- Bounds + maxiters breaks BFGS (SciML#508)

[Diff since v3.18.0](SciML/Optimization.jl@v3.18.0...v3.19.0)

[Diff since v3.17.0](SciML/Optimization.jl@v3.17.0...v3.18.0)

**Merged pull requests:**
- Support both float and array return extensions [Enzyme] (SciML#589) (@wsmoses)
- Setup tape compilation for ReverseDiff (SciML#590) (@ChrisRackauckas)
- Fix chunksize issue (SciML#595) (@Vaibhavdixit02)

**Closed issues:**
- Precompilation Warnings (SciML#579)
- Enzyme hessian throws a big error (SciML#592)

[Diff since v3.16.1](SciML/Optimization.jl@v3.16.1...v3.17.0)

**Merged pull requests:**
- [WIP] Enzyme and sparse updates (SciML#585) (@Vaibhavdixit02)
- Fix type instability in Enzyme extension (SciML#586) (@wsmoses)
- Bump actions/checkout from 3 to 4 (SciML#588) (@dependabot[bot])

[Diff since v3.16.0](SciML/Optimization.jl@v3.16.0...v3.16.1)

**Merged pull requests:**
- Fix typos in the extension names (SciML#582) (@ChrisRackauckas)
- Update extensions to not load packages directly only through Optimization (SciML#583) (@Vaibhavdixit02)

[Diff since v3.15.2](SciML/Optimization.jl@v3.15.2...v3.16.0)

**Closed issues:**
- Hessian coloring (SciML#269)
- Move all of the AD overloads to subpackages instead of Requires (SciML#309)
- Manual Hessian of Lagrangian function (SciML#343)
- `init` interface for reduced overhead in repeated solves (SciML#352)
- very easy to get bit by callbacks when there is a mismatch in the args of the callback function and the return of the loss  (SciML#538)
- Missing parameters in call to `instantiate_function` (SciML#559)
- Bump compat for Enzyme.jl to 0.11.2 or higher (SciML#564)
- Error with AutoZygote in OptimizationFunction after Julia version upgrade to 1.9.2 (SciML#571)

**Merged pull requests:**
- [Experimental] Add Sophia method implementation (SciML#534) (@Vaibhavdixit02)
- Handle sparse hessians, jacobians and hessvec product better (SciML#553) (@Vaibhavdixit02)
- CompatHelper: add new compat entry for Symbolics at version 5, (keep existing compat) (SciML#557) (@github-actions[bot])
- Try Enzyme 0.11.2 (SciML#561) (@Vaibhavdixit02)
- Some NLopt and MOI updates (SciML#562) (@Vaibhavdixit02)
- Update Project.toml to try Enzyme 0.11.4 and then 0.11.3 (SciML#563) (@Vaibhavdixit02)
- Update hessian implementation and avoid closure in gradient (SciML#565) (@Vaibhavdixit02)
- remove second copy of __moi_status_to_ReturnCode (SciML#568) (@visr)
- Refactor: correct a typo (SciML#569) (@tapyu)
- Throw error from callback true in NLopt to halt optimization (SciML#570) (@Vaibhavdixit02)
- CompatHelper: bump compat for ADTypes to 0.2, (keep existing compat) (SciML#574) (@github-actions[bot])
- CompatHelper: bump compat for NLopt to 1 for package docs, (keep existing compat) (SciML#575) (@github-actions[bot])
- CompatHelper: bump compat for NLopt to 1 for package OptimizationNLopt, (keep existing compat) (SciML#576) (@github-actions[bot])
- Add constraints support to ReverseDiff and Zygote (SciML#577) (@Vaibhavdixit02)
- Fix mtk empty contraints creation and update rosenbrock example (SciML#578) (@Vaibhavdixit02)

[Diff since v3.15.1](SciML/Optimization.jl@v3.15.1...v3.15.2)

**Merged pull requests:**
- Bring back AbstractRule in OptimizationOptimisers (SciML#552) (@Vaibhavdixit02)

[Diff since v3.15.0](SciML/Optimization.jl@v3.15.0...v3.15.1)

**Closed issues:**
- Document problems of more types than non-linear optimization problems only  (SciML#525)
- `OptimizationZygoteExt` and `OptimizationReversediffExt` broken  w/ Julia <1.9 (SciML#542)
- AutoForwardDiff is not recognized as AbstractADType (SciML#546)