Abstract: Repeater jamming is often the challenge faced by fire control radar (FCR). Considering the multi-stage confrontation scenario, a dwell time optimization method based on model-free reinforcement learning is proposed to handle the problem of radar beam multi-stage management with unknown environment model. Firstly, a Markov decision process with unknown environment model is built for multi-stage dwell time optimization. To evaluate the performance of radar detection, the expectation of the search to lock-on time of the FCR is selected as an evaluation criterion. Then, to overcome the challenge of the unknown environment model, a reinforcement learning framework for multi-stage dwell time optimization is formulated. According to the framework, a method of multi-stage dwell time optimization based on Q-learning is proposed. Finally, numerical results are provided to verify the validity of the proposed method.