22  Recipe: Load a DataFrame into PostgreSQL

22.1 Small or moderate datasets

df.to_sql(
    name="dataset",
    con=engine,
    schema="staging",
    if_exists="append",
    index=False,
    method="multi",
    chunksize=5000,
)

22.2 Validate the load

SELECT COUNT(*) AS loaded_rows
FROM staging.dataset;

22.3 Things to remember

  • Create the target table explicitly when data types matter.
  • Avoid replace in production unless dropping the table is intentional.
  • Compare source and target row counts.
  • Use PostgreSQL COPY for large datasets.