Orienteering and Reward Collection Problem

Literature review
Orienteering and reward collection problem (ORCP) refers to time-constrained
traveling activities that visit a subset of places in which each place has its own reward (Chao
et al., 1996). The objective of this problem is to maximize the obtained reward while
minimizing the travel distance. As a result, this problem is like a hybrid of the knapsack
problem (KP) and the traveling salesman problem (TSP). Due to restricted travel time, an agent
in an ORCP may not visit all locations. Finding the shortest path between two places/points
allows the agent to visit as many destinations as possible during the given travel time.
The orienteering problem varies in terms of number of agents, time window
characteristics, and time dependency of travel cost. The original ORCP can have a single agent
who travels along the route, or more than one agent, which in this case is known as team
orienteering problem (TOP). A point's time window describes its availability period, i.e. the
earliest and latest times an agent can visit at the point (Vansteenwegen et al., 2011). Modelling
ORCP with time dependency consideration means that the travel cost between two points
depends on the start moment (Fomin & Lingas, 2002) and this feature allows to calculate the
travel cost via multimodal public transportation (Gavalas et al., 2014).
(Zhang et al., 2018) presented a stochastic version of ORCP on a network of queues
where an agent may experience waiting before being able to serve in a location. The stochastic
waiting time in the problem is modeled as a Markov decision process (MDP) with two actions
at every deterministic time: stay at the location or go to the next point. The stay action might
be selected when the available start time of the next point is less than current time plus travel
time to that point. Due to high-dimensional space of the state, they took the rollout heuristic
algorithm from approximate dynamic programming (ADP) concept to approximate the
received reward at every decision epoch. (Reyes-Rubiano et al., 2020) discussed TOP with
position-dependent rewards. Rewards at each point change dynamically based on the order of
the visit, for example in a route, the beginning points to visit may have more reward than the
latter points. They modeled the problem as an integer programming (IP) and proposed biased-
randomized learn heuristic (BR-LH) to solve the problem. (Faigl, 2019) studied the ORCP but
in the application of data collection planning which include continuous optimization part. A
mixed integer non-linear programming (MINLP) approach is proposed to model four different
sub-problems: prize-collection TSP (PCTSP), close enough PCTSP, orienteering problem
(OP), and close enough OP. Their solution method based on an unsupervised learning approach
called growing self-organizing array (GSOA) has low computational requirements and gives
efficient route and effective reward collection. (Pěnička et al., 2019) also addressed data
collection planning problem but only considered unmanned aerial vehicles (UAVs) as agents.
They proposed three MINLP models and used variable neighborhood search (VNS)
metaheuristic to find not only the optimal visit sequence of all points, but also the optimal
heading angles of the UAVs, as these two decision variables will dictate the total travel distance
of the UAVs.
(Mukhina et al., 2019) provided open-source framework program for general OP where
an integer linear programming (ILP) model serves as a base for the framework. They
demonstrated the implementation of two heuristics: ant colony-based optimization (ACO) and
greedy-based recursive algorithm (GR) on four variants of OP: original OP, OP with
compulsory vertices (OPCV), OP with time window (OPTW), and OP with time dependency
(OPTD). (Dutta et al., 2020) constructed multi-objective OP where they assumed that the
ending point in the journey does not have to be the original problem, as in the original OP. Two
maximization goals are set: 1) maximizing customer satisfaction denoted by the more time
spending in a point, the more satisfied a customer will be. 2) maximizing the profit as usual
rewards in the OP. They used nondominated sorting genetic algorithm (NSGA) and strength
Pareto evolutionary algorithm (SPEA) to solve the multi-objective ILP problem. (Roozbeh et
al., 2020) introduced cooperative TOP with time window but with additional problem
requirements that rule specific routes must meet at some points and then come back to the
origin point if the agents have finished visiting all the points in the route. A mixed-integer
programming (MIP) is utilized to model the problem and solved by implementing adaptive
large neighborhood search (ALNS). (Xu et al., 2021) proposed approximation algorithms for
solving mobile chargers dispatching problem to recharge energy-critical sensor requirements.
They formulated the problem as integer non-linear programming (INLP) model with the
objective of maximizing the proposed submodular function. The submodular function
encourages the agents to visit new points in the sensor network. (Glock & Meyer, 2020)
introduced TOP with sampling locations in order to model tight-time UAV response problem
such as fire or chemical incident. They approached the problem with mixed-integer quadratic
programming (MIQP) formulation and proposed adaptive large neighborhood search with two-
phase multistart feature (2MLS).
(Kim et al., 2020) introduced multi-profit OP where this variant is an extension of OP
with time window unless there are more than one time window between two points. An MIP
model is proposed and simulated annealing (SA) heuristic is used for solving the model.
Furthermore, they formulated a visiting-time optimization algorithm (VTOA) for deciding
when the best time is for the agent to visit the points in a route. (Angelelli et al., 2021) studied
online version of the OP where the request points to visit arise first and the agent can
accept/reject the request. Then the accepted requests must be visited in a limited time while the
possible future request acceptance decision is made. The authors’ solution approach is first to
model the problem as MDP and then test different types of heuristic such as static
approximation algorithm, simple greedy (SG) algorithm, and sample average approximation
(SAA) algorithm. (Thayer & Carpin, 2021) considered stochasticity in OP. They include
probability of failure in the traveling cost and model it as an MDP which further solved by
linear programming as a starting point. A method for this stochastic OP called adaptive path
method (APM) is proposed to adapt to the changes occur while the agent is enroute, and to give
a policy to deviate from the preliminary plan. (Kobeaga et al., 2024) used branch-and-cut
algorithm for OP but in large-scale setting. The OP is modeled as ILP and the valid inequalities
in the problem are identified by column generation method. Valid inequalities in ILP are
constraints that always being satisfied by all integer feasible solutions and might exclude some
other non-integer solutions. The exact algorithm used in the problem gives optimal solutions
while most of the previous works approached the OP with heuristic algorithms. In conclusion,
Table 1 gives the articles summary related to OCPR with respect to model type used in the
study, number of agents in the problem, time window and time dependency considerations, and
the solution method.
 Table 1 Summary of OP articles
Reference Model Agent TW* TD* Solution
(Angelelli et al., 2021) MDP Single No No Approximation,
SG, SAA
(Faigl, 2019) MINLP Single No No GSOA
(Pěnička et al., 2019) MINLP Single No No VNS
(Dutta et al., 2020) ILP Single No No NSGA and SPEA
(Kobeaga et al., 2024) ILP Single No No Branch-and-cut
(Wu et al., 2024) IP Single No No Tabu search
(Vathis et al., 2023) IP Single No No Accommodation
planning algorithm
(Zhang et al., 2018) MDP Single Yes No Rollout algorithm
(Thayer & Carpin, 2021) MDP Single No Yes APM
(Le et al., 2022) N/A Single No Yes VNS
(Kim & Kim, 2022) MIP Single No Yes Dynamic
programming
(Kim et al., 2020) MIP Single Yes Yes SA and VTOA
(Mukhina et al., 2019) ILP Single Yes Yes ACO and GR
(Xu et al., 2021) INLP Multi No No Approximation
algorithm
(Glock & Meyer, 2020) MIQP Multi No No 2MLS
(Li et al., 2022) MIP Multi No No Dynamic
programming
(Fang et al., 2023) MIP Multi No No Large
neighborhood
searc
(Roozbeh et al., 2020) MIP Multi Yes No ALNS
(Reyes-Rubiano et al., 2020) IP Multi No Yes BR-L heuristic
(Panadero et al., 2022) MIP Multi No Yes BR-iterated local
search
*TW: Time window – TD: Time dependency – N/A: not available

REFERENCES
Angelelli, E., Archetti, C., Filippi, C., & Vindigni, M. (2021). A dynamic and probabilistic
orienteering problem. Computers & Operations Research, 136, 105454.
https://doi.org/10.1016/j.cor.2021.105454
Chao, I.-M., Golden, B. L., & Wasil, E. A. (1996). A fast and effective heuristic for the
orienteering problem. European Journal of Operational Research, 88(3), 475–489.
https://doi.org/10.1016/0377-2217(95)00035-6
Dutta, J., Barma, P. S., Mukherjee, A., Kar, S., & De, T. (2020). A multi-objective open set
orienteering problem. Neural Computing and Applications, 32(17), 13953–13969.
https://doi.org/10.1007/s00521-020-04798-7
Elzein, A., & Di Caro, G. A. (2022). A clustering metaheuristic for large orienteering problems.
PLOS ONE, 17(7), e0271751. https://doi.org/10.1371/journal.pone.0271751
Faigl, J. (2019). Data collection path planning with spatially correlated measurements using
growing self-organizing array. Applied Soft Computing, 75, 130–147.
https://doi.org/10.1016/j.asoc.2018.11.005
Fang, C., Han, Z., Wang, W., & Zio, E. (2023). Routing UAVs in landslides Monitoring: A
neural network heuristic for team orienteering with mandatory visits. Transportation
Research Part E: Logistics and Transportation Review, 175, 103172.
https://doi.org/10.1016/j.tre.2023.103172
Fomin, F. V., & Lingas, A. (2002). Approximation algorithms for time-dependent orienteering.
Information Processing Letters.
Gavalas, D., Konstantopoulos, C., Mastakas, K., & Pantziou, G. (2014). A survey on
algorithmic approaches for solving tourist trip design problems. Journal of Heuristics,
20(3), 291–328. https://doi.org/10.1007/s10732-014-9242-5
Glock, K., & Meyer, A. (2020). Mission Planning for Emergency Rapid Mapping with Drones.
Transportation Science, 54(2), 534–560. https://doi.org/10.1287/trsc.2019.0963
Kim, H., & Kim, B.-I. (2022). Hybrid dynamic programming with bounding algorithm for the
multi-profit orienteering problem. European Journal of Operational Research, 303(2),
550–566. https://doi.org/10.1016/j.ejor.2022.02.045
Kim, H., Kim, B.-I., & Noh, D. (2020). The multi-profit orienteering problem. Computers &
Industrial Engineering, 149, 106808. https://doi.org/10.1016/j.cie.2020.106808
Kobeaga, G., Rojas-Delgado, J., Merino, M., & Lozano, J. A. (2024). A revisited branch-and-
cut algorithm for large-scale orienteering problems. European Journal of Operational
Research, 313(1), 44–68. https://doi.org/10.1016/j.ejor.2023.07.034
Le, H. T., Middendorf, M., & Shi, Y. (2022). An Improvement Heuristic Based on Variable
Neighborhood Search for Dynamic Orienteering Problems with Changing Node Values
and Changing Budgets. SN Computer Science, 3(4), 326.
https://doi.org/10.1007/s42979-022-01205-x
Li, Y., Peyman, M., Panadero, J., Juan, A. A., & Xhafa, F. (2022). IoT Analytics and Agile
Optimization for Solving Dynamic Team Orienteering Problems with Mandatory Visits.
Mathematics, 10(6), 982. https://doi.org/10.3390/math10060982
Mukhina, K. D., Visheratin, A. A., & Nasonov, D. (2019). Orienteering Problem with
Functional Profits for multi-source dynamic path construction. PLOS ONE, 14(4),
e0213777. https://doi.org/10.1371/journal.pone.0213777
Panadero, J., Barrena, E., Juan, A. A., & Canca, D. (2022). The Stochastic Team Orienteering
Problem with Position-Dependent Rewards. Mathematics, 10(16), 2856.
https://doi.org/10.3390/math10162856
Pěnička, R., Faigl, J., Saska, M., & Váňa, P. (2019). Data collection planning with non-zero
sensing distance for a budget and curvature constrained unmanned aerial vehicle.
Autonomous Robots, 43(8), 1937–1956. https://doi.org/10.1007/s10514-019-09844-5
Reyes-Rubiano, L., Juan, A. A., Bayliss, C., Panadero, J., Faulin, J., & Copado, P. (2020). A
Biased-Randomized Learnheuristic for Solving the Team Orienteering Problem with
Dynamic Rewards. Transportation Research Procedia, 47, 680–687.
https://doi.org/10.1016/j.trpro.2020.03.147
Roozbeh, I., Hearne, J. W., & Pahlevani, D. (2020). A solution approach to the orienteering
problem with time windows and synchronisation constraints. Heliyon, 6(6), e04202.
https://doi.org/10.1016/j.heliyon.2020.e04202
Thayer, T. C., & Carpin, S. (2021). An Adaptive Method for the Stochastic Orienteering
Problem. IEEE Robotics and Automation Letters, 6(2), 4185–4192.
https://doi.org/10.1109/LRA.2021.3068699
Vansteenwegen, P., Souffriau, W., & Oudheusden, D. V. (2011). The orienteering problem: A
survey. European Journal of Operational Research, 209(1), 1–10.
https://doi.org/10.1016/j.ejor.2010.03.045
Vathis, N., Konstantopoulos, C., Pantziou, G., & Gavalas, D. (2023). The Vacation Planning
Problem: A multi-level clustering-based metaheuristic approach. Computers &
Operations Research, 150, 106083. https://doi.org/10.1016/j.cor.2022.106083
Wu, Q., He, M., Hao, J.-K., & Lu, Y. (2024). An effective hybrid evolutionary algorithm for
the clustered orienteering problem. European Journal of Operational Research, 313(2),
418–434. https://doi.org/10.1016/j.ejor.2023.08.006
Xu, W., Liang, W., Xu, Z., Peng, J., Peng, D., Liu, T., Jia, X., & Das, S. K. (2021).
Approximation Algorithms for the Generalized Team Orienteering Problem and its
Applications. IEEE/ACM Transactions on Networking, 29(1), 176–189.
https://doi.org/10.1109/TNET.2020.3027434
Zhang, S., Ohlmann, J. W., & Thomas, B. W. (2018). Dynamic Orienteering on a Network of
Queues. Transportation Science, 52(3), 691–706.
https://doi.org/10.1287/trsc.2017.0761