IPsec - does it support multicast
IPSec is a network security framework that is used to help protect IP traffic on the Network Layer of the OSI Model. When it was originally published in 1995, it didn't include support for multicast. Multicast support was always intended to be eventually added, but it wasn't a priority. Multicast support was finally added with newer RFCs in 2008 and 2009. The original and extended RFCs are included in the links below.
Although the designers of the protocol left room for the development of the protection of multicast traffic, the complete design was delayed, resulting in many IPsec implementations not supporting multicast. As far as Cisco goes, most implementations of IPsec have left out the protection of multicast.
There are however some Cisco devices that do support multicast over IPsec as defined by the newer RFCs using IPsec VTIs, including both IOS and Nexus devices. Check the capabilities of each individual device in the Cisco documentation to determine the capabilities of each.
For the most part, configuring multicast using IPsec can be tricky primarily because multicast traffic doesn't have a single, specific destination address, making it complex to manage and determine which security associations (SAs) to use.
If you're planning to deploy multicast over IPsec, it's essential to carefully consider the design and the devices you're using. Some devices might have better native support for IPsec multicast scenarios than others. It's also important to test thoroughly to ensure your solution is both secure and functional.
Typically, an alternative that supports multicast that is often used is a GRE tunnel configured in combination with IPsec.
Links:
https://datatracker.ietf.org/doc/html/rfc1827
https://datatracker.ietf.org/doc/html/rfc5374
https://datatracker.ietf.org/doc/html/rfc5406
https://www.cisco.com/en/US/technologies/tk583/tk372/technologies_white_paper0900aecd8029d629.html