Welcome to AstroPT’s documentation!

AstroPT 🐙: a Large Observation (foundation) Model for astronomy 🔭

Welcome to the documentation for AstroPT, a project for training astronomical large observation models. This repository began its life as Andrej Karpathy’s nanoGPT, and has been altered to be usable for astronomical observation data.

Check out the UniverseTBD Discord for updates: https://discord.gg/MNEVegvfJq

Contents:

• Introduction
  • What is AstroPT?
  • How does AstroPT work?
  • Key features
• Installation
  • Installation via pip
  • Installation from source using uv
• Quickstart
  • Loading a pre-trained model
  • Available pre-trained models
  • Basic model usage
• Training single and multimodal AstroPT models
  • Overview
  • Command-line usage
  • Key differences between scripts
  • Configuration options
  • Configuration files
  • Creating custom dataloaders
  • Output and monitoring
• Training with AION tokenisation
  • Overview
  • Key differences from patch-based tokenisation
  • Installation
  • Command-line usage
  • Modality configuration for AION
  • Dataset and dataloader setup
  • Loss function differences
  • Validation and visualisation
  • References
• Contributing
  • Where do I go from here?
  • Code of Conduct
  • How can I contribute?
  • If you can implement your feature
  • Development environment