Not sure why this thread went on when @griffin already hit the nail on the head in the first reply, in modem mode you can only use ONE port on the Hub.
With the Hub 3 in Modem mode any of the ports can be the one port used as the WAN port, just whichever one is grabbed by a device first for the DHCP reply. Once a device has received a DHCP served IP address from the Hub, any other devices connected to other ports on the hub will get nothing. So only one port is ever active on the Hub when in Modem mode. If you need to use more ports you have to use Router mode (and just disable all the wi-fi if you are using your own Wi-Fi like Orbi).
2nd key point is as you indicate... in Modem mode the Hub has an address of 192.168.100.1 in Router mode it changes to 192.168.0.1
So for those wanting to use an Orbi system for better wi-fi, just connect your main Orbi router to one of the ports on the Hub - DO NOT connect anything else to the Hub ports they won't work in Modem mode. You'll need to use the LAN ports on your Orbi's for connecting anything else. Note that in modem mode the Hub's DHCP process is kind of funky and can take a few minutes to serve an IP to your connected router. And note it WILL NOT be an IP address in the 192.168.100.x range (even though the Hub in Modem mode uses 192.168.100.1 as it's address).
The second option - if you want to use more ports on the Hub, is to leave the Hub in Router mode but just disable all the wi-fi on the Hub.
I've tried both options with my Hub 3 and Orbi system and both work... I preferred using Modem mode as I don't like the performance of the Hub in Router mode, but I've recently had to switch back to router mode as I wanted to connect a second router to my Hub to operate a separate work LAN - and you can only connect two routers to your Hub if it's in Router mode
