The success of differential evolution algorithm depends on its offspring breeding strategy and the associated control parameters. Improved Multi-Operator Differential Evolution (IMODE) proved its efficiency and ranked first in the CEC2020 competition. In this paper, an improved IMODE, called IMODEII, is introduced. In IMODEII, Reinforcement Learning (RL), a computational methodology that simulates interaction-based learning, is used as an adaptive operator selection approach. RL is used to select the best-performing action among three of them in the optimization process to evolve a set of solution based on the population state and reward value. Different from IMODE, only two mutation strategies have been used in IMODEII. We tested the performance of the proposed IMODEII by considering 12 benchmark functions with 10 and 20 variables taken from CEC2022 competition on single objective bound constrained numerical optimisation. A comparison between the proposed IMODEII and the state-of-the-art algorithms is conducted, with the results demonstrating the efficiency of the proposed IMODEII.