We recently have been developing designing methods of nonlinear filters by learning. In this paper, our method is extended to morphological filters with grayscale structuring elements. The basic morphological operations are defined by shifting and overlapping of images; the morphological operations by grayscale structuring elements are realized by shifting image not only along the pixel coodinate but also along the direction of brightness. This class of filters is proved to construct all increasing shift-invariant filters; theoretically linear filters can be realized. To optimize grayascale structuring element, the simulated annealing method are modified and utilized.