quantization-5f46c11
Pre-release
Pre-release
Half way, in progress.
add quantization types, f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0, q2_k, q3_k, q4_k, q5_k, q6_k, iq2_xxs, iq2_xs, iq3_xxs, iq1_s, iq4_nl, iq3_s, iq2_s, iq4_xs, i8, i16, i32, i64, f64, iq1_m, bf16, q4_0_4_4, q4_0_4_8, q4_0_8_8.
ElMiloPy, thank you.