Abstract: The maximum likelihood estimation of DOA is an appealing algorithm with the best statistical performance than that of others. However, this method requires optimizing of multi-dimensional parameter which results in large computation quantity. In view of that this paper suggests to use the modified simplex algorithm to optimize the parameter. The paper firstly uses the array output eovarianee matrix to obtain the eoame DOA estimation and form the initial simplex with the coarse estimation as its center of gravity. Because all the parameters have certain value range, so we transform the optimization with eonstaint into optimization without it by defining a penalty function. Finally we unify the penalty function and modified simplex algorithm to obtain the DOA estimation. The computer simulation result shows the correctness and efficiency of the algorithm.