OLMo-1B-0724-hf

We updated our OLMo 7B model with some small improvements to training data and annealing techniques. A pretty minor bump, but we have some exciting new models coming soon. An interesting thing I've learned about pretraining is how loss spikes often relate to "skipped tokens," making the models worse at a fixed compute budget. This is because the gradients is loss spikes get clipped and the data effectively does nothing.