Adapting free-space fast multipole method for layered media Green's function: Algorithm and analysis

We present a numerical algorithm for an accurate and efficient computation of the convolution of the frequency domain layered media Green's function with a given density function. Instead of compressing the convolution matrix directly as in the classical fast multipole, fast direct solver, and H-matrix algorithms, this new algorithm considers a translated form of the original matrix so existing blocks from the highly optimized free-space fast multipole method can be easily adapted to the layered media Green's function. An asymptotic analysis is performed on the Sommerfeld integrals to provide an estimate of the decay rate in the new “multipole” and “local” expansions. To avoid the highly oscillatory integrand in the original integral representations when the source and target are close to each other, mathematically equivalent alternative direction integral representations are introduced. The convergence of the new expansions and quadrature rules for the original and alternative direction representations are numerically validated.