pebkac routing problem

[Martin P. Hellwig]

Hi all,

I am having a routing problem.
At my work site I am switching between 2 networks (from a cable to fiber 
connection) and registrar and DNS.

To make the move as transparent as possible for my mail server I want to 
configure both networks at the same time. Then set the mx to the new IP 
and after a couple of days remove the old adres.
Both network are reachable from the same physical network.

So I tried it first on my test server (which already is on the fiber 
network but reversing the process to move to the cable IP should 
logically be the same of course I tried it both ways but did not write 
down that progress, what I did  write down is placed beneath all).

My conclusion where that the package always return via the default 
gateway and thus get blocked by the next hop gateway which (reasonable) 
blocks routing for foreign IP's.

My question is, how can I configure my BSD box, that a IP package is 
always returned to the gateway it came from when?

Thanks in advance for any suggestions, directions or any other comment.

Martin (mph)



current configuration:
145.103.249.189/27	gw 145.103.249.190
appending configuration:
213.126.48.226/24 	gw 213.126.48.1
inet4 Routing tables before fiddling:
>
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            190-249-103-145.cu UGSc        2        8   fxp0
localhost          localhost          UH          0        0    lo0
160-249-103-145.cu link#1             UC          1        0   fxp0
190-249-103-145.cu 00:e0:2b:00:00:84  UHLW        3        0   fxp0   1169
>
# ifconfig fxp0 alias 213.126.48.226/24
# ifconfig fxp0
>
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	inet 145.103.249.189 netmask 0xffffffe0 broadcast 145.103.249.191
	inet6 fe80::230:6eff:fe05:fdcc%fxp0 prefixlen 64 scopeid 0x1
	inet 213.126.48.226 netmask 0xffffff00 broadcast 213.126.48.255
	ether 00:30:6e:05:fd:cc
	media: Ethernet autoselect (100baseTX <full-duplex>)
	status: active
>
Routing tables now:
>
Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            190-249-103-145.cu UGSc        2        8   fxp0
localhost          localhost          UH          0        0    lo0
160-249-103-145.cu link#1             UC          1        0   fxp0
190-249-103-145.cu 00:e0:2b:00:00:84  UHLW        3        0   fxp0    776
213.126.48.0.ip.on link#1             UC          1        0   fxp0
213.126.48.224.ip. 00:01:02:65:34:4a  UHLW        0        6   fxp0   1176
>
Testing from 213.126.48.224 (WORKS):
# ping -t1 213.126.48.226
PING 213.126.48.226 (213.126.48.226): 56 data bytes
64 bytes from 213.126.48.226: icmp_seq=0 ttl=64 time=0.380 ms
--- 213.126.48.226 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
Testing from xs3.xs4all.nl (FAILS):
# ping -t1 213.126.48.226
PING 213.126.48.226 (213.126.48.226): 56 data bytes
--- 213.126.48.226 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
Taking a look at the tcpdump (THERE IS ECHO REPLY):
84) xs3.xs4all.nl > 213.126.48.226.ip.onderwijs.casematelecom.nl: icmp 
64: echo request seq 0
21:07:35.394053 IP (tos 0x0, ttl  64, id 3149, offset 0, flags [none], 
length: 84) 213.126.48.226.ip.onderwijs.casematelecom.nl > 
xs3.xs4all.nl: icmp 64: echo reply seq 0

So the outgoing echo reply is probably routed through default gateway 
and there silently discarded because of IP policies.

Trying to add a route for 213.126.48.0/24 (FAILS):
# route add 213.126.48.0/24 213.126.48.1
route: writing to routing socket: File exists
add net 213.126.48.0: gateway 213.126.48.1: File exists