Ok I see the only way they can do that is when your modem is providing NAT to all the devices so the public IP originates from that very modem, if modem is in the bridge mode and other device behind it receives the public ip then they will never see the modem but your computer instead.

On my network, modem is just a gateway where my Linux server gets the public ip and then serves internet to all the computers on the network, so I can spoof MAC on my server and itll also get the new IP, now any computer on the network wants to get out it goes through server which IP and MAC is visible to the world.

It's also possible to spoof your modems address but that requires lots more patience as it's usually locked up and you need to use different tricks plus custom software to trick it to download your software which then wrecks havock with the modem. Anything is possible with enough of skill and patience, I do this for a living actually.