[Diff since v3.19.0](SciML/Optimization.jl@v3.19.0...v3.19.1)

**Merged pull requests:**
- map `maxiters` to `outer_iterations` (SciML#544) (@SebastianM-C)
- Eliminate some runtime dispatch and other things (SciML#597) (@Vaibhavdixit02)

**Closed issues:**
- Bounds + maxiters breaks BFGS (SciML#508)

[Diff since v3.18.0](SciML/Optimization.jl@v3.18.0...v3.19.0)

[Diff since v3.17.0](SciML/Optimization.jl@v3.17.0...v3.18.0)

**Merged pull requests:**
- Support both float and array return extensions [Enzyme] (SciML#589) (@wsmoses)
- Setup tape compilation for ReverseDiff (SciML#590) (@ChrisRackauckas)
- Fix chunksize issue (SciML#595) (@Vaibhavdixit02)

**Closed issues:**
- Precompilation Warnings (SciML#579)
- Enzyme hessian throws a big error (SciML#592)

[Diff since v3.16.1](SciML/Optimization.jl@v3.16.1...v3.17.0)

**Merged pull requests:**
- [WIP] Enzyme and sparse updates (SciML#585) (@Vaibhavdixit02)
- Fix type instability in Enzyme extension (SciML#586) (@wsmoses)
- Bump actions/checkout from 3 to 4 (SciML#588) (@dependabot[bot])

[Diff since v3.16.0](SciML/Optimization.jl@v3.16.0...v3.16.1)

**Merged pull requests:**
- Fix typos in the extension names (SciML#582) (@ChrisRackauckas)
- Update extensions to not load packages directly only through Optimization (SciML#583) (@Vaibhavdixit02)

[Diff since v3.15.2](SciML/Optimization.jl@v3.15.2...v3.16.0)

**Closed issues:**
- Hessian coloring (SciML#269)
- Move all of the AD overloads to subpackages instead of Requires (SciML#309)
- Manual Hessian of Lagrangian function (SciML#343)
- `init` interface for reduced overhead in repeated solves (SciML#352)
- very easy to get bit by callbacks when there is a mismatch in the args of the callback function and the return of the loss  (SciML#538)
- Missing parameters in call to `instantiate_function` (SciML#559)
- Bump compat for Enzyme.jl to 0.11.2 or higher (SciML#564)
- Error with AutoZygote in OptimizationFunction after Julia version upgrade to 1.9.2 (SciML#571)

**Merged pull requests:**
- [Experimental] Add Sophia method implementation (SciML#534) (@Vaibhavdixit02)
- Handle sparse hessians, jacobians and hessvec product better (SciML#553) (@Vaibhavdixit02)
- CompatHelper: add new compat entry for Symbolics at version 5, (keep existing compat) (SciML#557) (@github-actions[bot])
- Try Enzyme 0.11.2 (SciML#561) (@Vaibhavdixit02)
- Some NLopt and MOI updates (SciML#562) (@Vaibhavdixit02)
- Update Project.toml to try Enzyme 0.11.4 and then 0.11.3 (SciML#563) (@Vaibhavdixit02)
- Update hessian implementation and avoid closure in gradient (SciML#565) (@Vaibhavdixit02)
- remove second copy of __moi_status_to_ReturnCode (SciML#568) (@visr)
- Refactor: correct a typo (SciML#569) (@tapyu)
- Throw error from callback true in NLopt to halt optimization (SciML#570) (@Vaibhavdixit02)
- CompatHelper: bump compat for ADTypes to 0.2, (keep existing compat) (SciML#574) (@github-actions[bot])
- CompatHelper: bump compat for NLopt to 1 for package docs, (keep existing compat) (SciML#575) (@github-actions[bot])
- CompatHelper: bump compat for NLopt to 1 for package OptimizationNLopt, (keep existing compat) (SciML#576) (@github-actions[bot])
- Add constraints support to ReverseDiff and Zygote (SciML#577) (@Vaibhavdixit02)
- Fix mtk empty contraints creation and update rosenbrock example (SciML#578) (@Vaibhavdixit02)

[Diff since v3.15.1](SciML/Optimization.jl@v3.15.1...v3.15.2)

**Merged pull requests:**
- Bring back AbstractRule in OptimizationOptimisers (SciML#552) (@Vaibhavdixit02)

[Diff since v3.15.0](SciML/Optimization.jl@v3.15.0...v3.15.1)

**Closed issues:**
- Document problems of more types than non-linear optimization problems only  (SciML#525)
- `OptimizationZygoteExt` and `OptimizationReversediffExt` broken  w/ Julia <1.9 (SciML#542)
- AutoForwardDiff is not recognized as AbstractADType (SciML#546)

[Diff since v3.14.2](SciML/Optimization.jl@v3.14.2...v3.15.0)

**Closed issues:**
- InvalidSystemException error in Rosetta-OPF on latest releases (SciML#537)
- Have to specify derivatives for Optim.ZerothOrderOptimizer with bounds (SciML#548)

**Merged pull requests:**
- [WIP] Misc docs updates (SciML#513) (@Vaibhavdixit02)
- Add cache dispatch to Enzyme `instantiate_function` (SciML#535) (@Vaibhavdixit02)
- Add cache interface to all wrapper packages (SciML#536) (@Vaibhavdixit02)
- Do not ignore the initial guess in OptimizationBBO (SciML#540) (@SebastianM-C)
- Try running CI on 1.8 (SciML#545) (@Vaibhavdixit02)
- Try getting BBO tests passing (SciML#547) (@Vaibhavdixit02)
- Format master (SciML#549) (@Vaibhavdixit02)
- Bump versions (SciML#550) (@Vaibhavdixit02)

[Diff since v3.14.1](SciML/Optimization.jl@v3.14.1...v3.14.2)

**Closed issues:**
- Document `EnzymeAD` (SciML#522)

**Merged pull requests:**
- Remove Nonconvex from docs for now (SciML#517) (@baggepinnen)
- prefer use of lagragian hessian if available (SciML#518) (@baggepinnen)
- Performance Improvements for OptimizationBBO (SciML#519) (@Zentrik)
- Update OptimizationGCMAES.jl to set `allowscallbacks` to false (SciML#527) (@Vaibhavdixit02)
- generate `lag_h` function for FiniteDiff (SciML#529) (@baggepinnen)
- Move AD files to extension packages (SciML#530) (@Vaibhavdixit02)
- CompatHelper: add new compat entry for ADTypes at version 0.1, (keep existing compat) (SciML#532) (@github-actions[bot])
- Remove buggy `structural_simplify` usage (SciML#539) (@YingboMa)