Spring batch

Spring Batch

Spring Boot 3.x ships Spring Batch 5. The API changed significantly from 4.x — most online examples are wrong. The two rules that break the most agent-generated code:

1. Do NOT add @EnableBatchProcessing. Boot auto-configures the JobRepository, JobLauncher, and transaction manager. Adding @EnableBatchProcessing disables that auto-configuration and you lose all the wired beans.
2. JobBuilderFactory and StepBuilderFactory are gone. Build jobs and steps with new JobBuilder(name, jobRepository) / new StepBuilder(name, jobRepository), injecting the JobRepository and PlatformTransactionManager Boot already created.

Job & Step (Spring Batch 5 API)

@Configuration
@RequiredArgsConstructor
public class OrderExportJobConfig {

    @Bean
    public Job orderExportJob(JobRepository jobRepository, Step exportStep) {
        return new JobBuilder("orderExportJob", jobRepository)
            .incrementer(new RunIdIncrementer()) // lets the same job be re-run; see "Idempotency"
            .start(exportStep)
            .build();
    }

    @Bean
    public Step exportStep(JobRepository jobRepository,
                           PlatformTransactionManager txManager, // Boot's, injected — do NOT new one up
                           ItemReader<Order> reader,
                           ItemProcessor<Order, OrderRow> processor,
                           ItemWriter<OrderRow> writer) {
        return new StepBuilder("exportStep", jobRepository)
            .<Order, OrderRow>chunk(500, txManager) // chunk size is the commit interval — and a TX boundary
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .faultTolerant()
            .skip(FlatFileParseException.class)
            .skipLimit(50)
            .build();
    }
}

chunk(500, txManager) means: read 500 items, process each, hand the list of 500 to the writer, commit one transaction, repeat. The chunk is the unit of restart and the unit of rollback.

Idempotency & Restartability — the #1 operational gotcha

A JobInstance is identified by its identifying JobParameters. Launch the same job with the same identifying parameters twice and you get:

JobInstanceAlreadyCompleteException: A job instance already exists and is complete

This is by design — Batch refuses to re-run completed work. Two ways to handle it:

// Option A — RunIdIncrementer on the job (above) + JobLauncherApplicationRunner bumps run.id each launch.
// Option B — add a unique identifying parameter yourself when launching:
JobParameters params = new JobParametersBuilder()
    .addString("status", "COMPLETED")              // identifying — part of the instance key
    .addLong("run.id", System.currentTimeMillis()) // identifying & unique — makes each run a new instance
    .toJobParameters();

Mark a parameter non-identifying with the false flag when it's metadata that shouldn't change the instance identity (e.g. a request id you log but don't key on):

.addString("requestId", requestId, false) // non-identifying — excluded from the instance key

A failed job, by contrast, is resumed when relaunched with the same parameters — it skips completed steps and restarts the failed step from the last committed chunk. That is the point of the metadata tables. Don't defeat it by always passing a unique parameter if you want resume-on-failure.

ItemReader — sort key and thread-safety

@Bean
@StepScope // required: late-binds jobParameters at step execution, not context startup
public JpaPagingItemReader<Order> orderReader(
        EntityManagerFactory emf,
        @Value("#{jobParameters['status']}") String status) {
    return new JpaPagingItemReaderBuilder<Order>()
        .name("orderReader")
        .entityManagerFactory(emf)
        .queryString("SELECT o FROM Order o WHERE o.status = :status ORDER BY o.id") // ORDER BY is MANDATORY
        .parameterValues(Map.of("status", OrderStatus.valueOf(status)))
        .pageSize(500) // keep pageSize == chunk size
        .build();
}

• Paging readers require a deterministic ORDER BY on a unique column. Without it the DB returns rows in arbitrary order across pages → rows get skipped or processed twice. This is silent data corruption, not an error.
• JdbcCursorItemReader is NOT thread-safe. JdbcPagingItemReader / JpaPagingItemReader are safe for multi-threaded steps. For a non-thread-safe reader in a multi-threaded step, wrap it in SynchronizedItemStreamReader.
• Don't mutate the column you page on inside the same job. If the writer flips status from PENDING to DONE while the reader pages WHERE status = 'PENDING' ORDER BY id, the result set shifts under you and pages are missed. Read into a stable snapshot, page by immutable id, or use a cursor reader.

ItemProcessor — returning null filters

@Component
public class OrderProcessor implements ItemProcessor<Order, OrderRow> {
    @Override
    public OrderRow process(Order order) {
        if (order.getTotal().isZero()) {
            return null; // ⚠️ null = FILTER this item; it is NOT written and NOT an error
        }
        return OrderRow.from(order);
    }
}

Returning null silently drops the item from the chunk. That's a feature (filtering) but a footgun if you returned null by accident expecting it to pass through.

ItemWriter — Spring Batch 5 signature change

In 5.x the writer receives a Chunk<? extends T>, not List<? extends T>:

@Override
public void write(Chunk<? extends OrderRow> chunk) { // was List<? extends T> in 4.x
    repository.saveAll(chunk.getItems());
}

For SQL writes, prefer the batched JDBC writer over per-row saves — it uses one addBatch():

@Bean
public JdbcBatchItemWriter<OrderRow> orderWriter(DataSource dataSource) {
    return new JdbcBatchItemWriterBuilder<OrderRow>()
        .dataSource(dataSource)
        .sql("INSERT INTO order_export (id, total) VALUES (:id, :total)")
        .beanMapped()
        .build();
}

The writer runs inside the chunk transaction. Never fire emails, publish to Kafka, or call webhooks from a writer — if the chunk rolls back you've already sent it. Bind side effects to the job completion instead (see [[transactional-patterns]] and the listener below).

Launching jobs

Boot runs every Job bean on startup by default. For scheduled or on-demand jobs, turn that off and launch explicitly:

spring:
  batch:
    job:
      enabled: false           # don't run jobs on app startup; we trigger them ourselves
    jdbc:
      initialize-schema: never # manage BATCH_* tables with Flyway in prod (see below)

@Component
@RequiredArgsConstructor
public class OrderExportScheduler {

    private final JobLauncher jobLauncher;
    private final Job orderExportJob;

    @Scheduled(cron = "0 0 2 * * *")
    public void runNightly() throws JobExecutionException {
        JobParameters params = new JobParametersBuilder()
            .addString("status", "COMPLETED")
            .addLong("run.id", System.currentTimeMillis())
            .toJobParameters();
        jobLauncher.run(orderExportJob, params);
    }
}

The default JobLauncher is synchronous (SyncTaskExecutor) — jobLauncher.run(...) blocks the @Scheduled thread until the whole job finishes. For fire-and-forget, configure the launcher with an async TaskExecutor, or trigger from a request thread only if you accept the block.

Metadata schema in production

Spring Batch needs its BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, BATCH_STEP_EXECUTION, … tables. initialize-schema: always is fine for dev/embedded DBs but don't let Batch DDL your production database on startup. Set initialize-schema: never and ship the schema as a versioned [[flyway-migrations]] migration (the canonical DDL lives in org/springframework/batch/core/schema-*.sql inside spring-batch-core).

Listeners — work that must run after the job, not per chunk

@Bean
public Job orderExportJob(JobRepository jobRepository, Step exportStep) {
    return new JobBuilder("orderExportJob", jobRepository)
        .incrementer(new RunIdIncrementer())
        .listener(new JobExecutionListener() {
            @Override public void afterJob(JobExecution exec) {
                if (exec.getStatus() == BatchStatus.COMPLETED) {
                    notifier.notifyExportReady(exec.getJobParameters()); // safe: all chunks committed
                }
            }
        })
        .start(exportStep)
        .build();
}

Don't reach for Batch when you don't need it

Spring Batch earns its complexity (metadata tables, restart, chunking) on large, restartable, auditable bulk jobs. For a quick one-off async task, @Async or a @Scheduled loop is lighter. For durable background jobs with retry, a job queue is a better fit. Match the tool to the scale.

Gotchas

• Agent adds @EnableBatchProcessing — on Boot 3 it disables auto-config; remove it, just inject JobRepository
• Agent uses JobBuilderFactory / StepBuilderFactory — removed in Batch 5; use new JobBuilder(name, repo) / new StepBuilder(name, repo)
• Agent calls .chunk(500) without a transaction manager — Batch 5 requires .chunk(500, txManager)
• Agent writes write(List<? extends T> items) — Batch 5 signature is write(Chunk<? extends T> chunk)
• Agent re-runs a job with identical parameters and hits JobInstanceAlreadyCompleteException — add RunIdIncrementer or a unique identifying param
• Agent adds a unique param every run on a job that should resume-on-failure — kills restartability; only add it when you want a fresh instance
• Agent writes a paging reader query with no ORDER BY (or a non-unique one) — pages skip/duplicate rows silently; order by a unique column
• Agent uses JdbcCursorItemReader in a multi-threaded step — not thread-safe; use a paging reader or SynchronizedItemStreamReader
• Agent pages on a column the writer mutates in the same job — result set shifts; page on an immutable id
• Agent returns null from a processor expecting pass-through — null filters (drops) the item
• Agent sends email / publishes events from the ItemWriter — runs inside the chunk TX; do it in an afterJob listener
• Agent forgets @StepScope on a reader that reads jobParameters — @Value("#{jobParameters[...]}") only binds in step scope
• Agent leaves jobs running on startup in a web app — set spring.batch.job.enabled=false and launch explicitly
• Agent lets initialize-schema: always DDL the prod DB — use never + a Flyway migration for the BATCH_* tables