So I took the heap dump, analyzed it, and saw some Spring Batch classes that were taking huge amount of my heap.
I googled around and it seems it's not very common to use Spring batch in-memory mode (which is done by registering MapJobRepositoryFactoryBean as job-repository), and OutOfMemory is a quite often problem in those rare cases. Seems like the MapJobRepositoryFactoryBean keeps the track of all your jobs, data, job history and who knows what else in memory. So even with not so complicated jobs, eventually you get the OOM. MapJobRepositoryFactoryBean has some nice clear methods that could be used to clear the data, but how to access it? Not so obvious, so here's the way I did it. If anybody has better idea, please share :)
First, let's create our own JobRepositoryFactoryBean, that will give us access to clear method
public class MyMapJobRepositoryFactoryBean extends MapJobRepositoryFactoryBean {The CleanableJobRepository is a simple interface
@Override
public Object getObject() throws Exception {
return new MySimpleJobRepository(createJobInstanceDao(), createJobExecutionDao(), createStepExecutionDao(), createExecutionContextDao());
}
public class MySimpleJobRepository extends SimpleJobRepository implements CleanableJobRepository {
MySimpleJobRepository(final JobInstanceDao jobInstanceDao,
final JobExecutionDao jobExecutionDao,
final StepExecutionDao stepExecutionDao,
final ExecutionContextDao ecDao) {
super(jobInstanceDao, jobExecutionDao, stepExecutionDao, ecDao);
}
@Override
public void clean() {
((MapJobInstanceDao) getJobInstanceDao()).clear();
((MapJobExecutionDao) getJobExecutionDao()).clear();
((MapStepExecutionDao) getStepExecutionDao()).clear();
((MapExecutionContextDao) getExecutionContextDao()).clear();
}
}
}
public interface CleanableJobRepository {Now we can use it as our job repository
void clean();
}
<bean id="jobRepository"The last thing is to do the actual cleanup. Since I already run several jobs, I simply added a new one, that does the cleanup
class="nadircode.MyMapJobRepositoryFactoryBean" />
public class BatchExecutionContextCleanupJob extends QuartzJobBase implements StatefulJob {Simply register the job in your scheduler, and your memory will be cleaned.
private CleanableJobRepository jobRepository;
@Override
protected void executeInternal(final JobExecutionContext arg0) throws JobExecutionException {
try {
@SuppressWarnings("unchecked")
List<JobExecutionContext> currentlyExecutingJobs = scheduler.getCurrentlyExecutingJobs();
if (onlyThisJobIsRunning(currentlyExecutingJobs)) {
jobRepository.clean();
}
} catch (SchedulerException ex) {
throw new JobExecutionException(ex);
}
}
//make sure that only this job is running, to not to clear data that still may be used by other jobs!
private boolean onlyThisJobIsRunning(final List<JobExecutionContext> currentlyExecutingJobs) {
if (currentlyExecutingJobs.size() > 1) {
return false;
}
JobExecutionContext jobExecutionContext = currentlyExecutingJobs.get(0);
return jobExecutionContext.getJobDetail().getJobClass().equals(this.getClass());
}
public void setJobRepository(final CleanableJobRepository jobRepository) {
this.jobRepository = jobRepository;
}
}
QuartzJobBase is my own QuartzJob extension that gives you access to scheduler.
Thanks a lot. Is there a way to clear data related to specific job only ? Say if I have the jobId of the job can I clear all the data related with it ?
OdpowiedzUsuńHi, I never tried, and not using this now so it's hard for me to check. But maybe you should inspect the code of MapJobExecutionDao and related, maybe there is something that will allow to remove only specified job data.
UsuńThanks a lot ... I was facing same issue from few days. This Solution resolved the Problem.
OdpowiedzUsuńThanks again.
I had troubled with a memory leak. Thank you from Japan.
OdpowiedzUsuń