Optimising the optimiser: accelerating genetic programming for smarter scheduling

REANNZ helped researchers optimise AI scheduling code, cutting runtime from 1,298 to 195 seconds and enabling larger experiments with fewer resources.

Research background

Yuan Tian and Qianhui Wang are PhD students at Te Herenga Waka—Victoria University of Wellington researching advanced optimisation algorithms for solving complex resource-constrained project scheduling problems. Their work focuses on developing AI-driven machine learning methods that can be applied to a wide range of planning scenarios.

One important application is medical dispatch, where ambulances and emergency response teams must be allocated to incidents while balancing constraints such as limited personnel, equipment availability, travel times, and treatment priorities. Faster and more effective scheduling decisions have the potential to improve emergency response times and patient outcomes. Beyond healthcare, similar optimisation techniques can benefit industries such as logistics, manufacturing, construction, and transportation by reducing planning time and improving operational efficiency.

Project challenges

The researchers' software is implemented in Python and uses genetic programming hyper-heuristics—a machine learning approach that evolves the scheduling rules themselves rather than learning fixed model parameters.

This flexibility enables the algorithm to discover highly effective scheduling strategies but comes at a significant computational cost, as evaluating thousands of candidate heuristics requires repeatedly solving large scheduling problems. As a result, execution times had become a major limitation to experimentation and algorithm development.

What was done

REANNZ worked with the researchers to profile the application, identify performance bottlenecks, and implement a series of targeted optimisations using a representative workload. The main improvements included:

  • Running the Python interpreter in optimised mode (-O) to eliminate unnecessary runtime overhead. Replacing NetworkX graph views with a precomputed dictionary representation, reducing graph access overhead and delivering a 21% performance improvement.
  • Redesigning resource feasibility checking to update resource usage incrementally rather than recomputing it from scratch for every scheduling decision, resulting in a further 25% improvement.
  • Refactoring the earliestStart routine so it could be accelerated with Numba's just-in-time compilation, requiring careful restructuring to use numba’s supported data types and yielding an additional 14% speedup.

 

diagram of a decision-making tree

The above figure shows how profiling can help identify performance hotspots, which should always be a starting point when trying to improve computational efficiency. The colour represents the percentage of time spent in any given function, excluding calls to other functions. The largest bottleneck here is the “_compute_dynamic_cpm” function, where 36% of the execution time is consumed.

 

Main outcomes

The combined optimisations reduced the execution time of the representative test case from 1,298 seconds to 195 seconds—a 6.6× speedup. The substantial reduction in runtime will enable the researchers to evaluate many more candidate scheduling heuristics in the same amount of time, and save up to 3.4 million core-hours on REANNZ’s high performance platform.

Researcher feedback

This project has been a rare and highly valuable experience. Before working with REANNZ, we had been struggling with the computational efficiency of our research code for a long time. The consultancy provided both the expertise and the momentum we needed to address these challenges systematically. Communication throughout the project was highly effective.

We are especially grateful to Alex and Thomas for taking the time to understand our research, listening carefully to our explanations, and being willing to explore a complex and sometimes untidy research codebase to identify the key performance bottlenecks. The collaboration has delivered substantial improvements to the efficiency of our software and will allow us to conduct larger and more comprehensive experiments within the same computational budget. Just as importantly, we have gained valuable practical knowledge about profiling and code optimisation. We hope to share these experiences with the wider research community so that other researchers facing similar computational challenges can also benefit.

- Yuan Tian, Te Herenga Waka—Victoria University of Wellington

 


 

This case study shares some of the technical details and outcomes provided through our Consultancy Service. This service supports projects across a range of domains, with an aim to lift researchers’ productivity, efficiency, and skills in research computing. Get in touch to discuss how our Research Software Engineers and specialist support could help advance your project.

Find anything about our products, services, and more. Enter a query in the search input above.