The ESA Planck satellite has begun studying the anisotropies of the cosmic microwave background radiation over the whole sky with unprecedented sensitivity and high angular resolution. The High Frequency Instrument, HFI, on Planck is observing simultaneously in six bands in the range 100 GHz to 857 GHz. The inclusion of non-CMB bands allows for robust removal of foreground sources from the data. This paper is concerned with the design, modeling and predicted performances of the two highest frequency channels centered on 545 GHz and 857 GHz, which use specialized multi-mode feedhorns, and are dedicated to observing these foregrounds. Multi-mode systems have the advantage of increasing the throughput, and thus sensitivity, of the detection assembly when diffraction limited resolution is not required. The horns are configured in a back-to-back setup which transmits the signal through filters to a detector horn. The modeling of the broadband beam patterns on the sky is shown to require careful analysis. Simulations of the complex interactions of the horns is computationally challenging when the detector horn in the relay system is included. The paper describes the approach to modeling these high frequency channels and discusses how the optical requirements on the horn designs are met in terms of spillover, edge taper, illumination of the telescope aperture and beam patterns on the sky.