GMail fails SPF checks on POP3 fetched messages

spf-logo-midi[1]

It seems that, under certain conditions, GMail reports failed SPF checks for messages fetched via POP3 from other mail servers.

I noticed this behaviour on messages received, for example, by mail servers where an internal relay is used, like the following message sent from PayPal (which uses an hard-fail policy):

Delivered-To: MYSELF@gmail.com
...
Received-SPF: fail (google.com: domain of xxxyyyzzz@emea.e.paypal.com does not
        designate A.B.C.D as permitted sender) client-ip=A.B.C.D;
Received: by 10.64.225.172 with POP3 ...
X-Gmail-Fetch-Info: MYSELF@MYDOMAIN.TLD 3 pop3.MYDOMAIN.TLD
        995 MYSELF@MYDOMAIN.TLD
Return-Path: <xxxyyyzzz@emea.e.paypal.com>
Delivered-To: MYSELF@MYDOMAIN.TLD
Received: from server1.MYPROVIDER.TLD (A.B.C.D)
        by server2.MYPROVIDER.TLD with SMTP; ...
Received: from outbound.emea.e.paypal.com (96.47.30.179)
        by mx1.MYPROVIDER.TLD with SMTP; ...
Return-Path: <xxxyyyzzz@emea.e.paypal.com>
...
From: "PayPal" <paypal@e.paypal.it>
To: MYSELF@MYDOMAIN.TLD

This is the SPF record for emea.e.paypal.com:

;; QUESTION SECTION:
;emea.e.paypal.com.        IN   TXT
 
;; ANSWER SECTION:
emea.e.paypal.com.   3600  IN   TXT  "v=spf1 a:outbound.emea.e.paypal.com -all"

It authorizes every IP address resolved by the outbound.emea.e.paypal.com A record.

In the example, MyProvider receives the message from 96.47.30.179, which is one of the many IP addresses resolved by and authorized by outbound.emea.e.paypal.com:

;; QUESTION SECTION:
;outbound.emea.e.paypal.com.       IN   A
 
;; ANSWER SECTION:
outbound.emea.e.paypal.com.  300   IN	A   96.47.30.224
...
outbound.emea.e.paypal.com.  300   IN	A   96.47.30.179

After receiving the message, MyProvider uses an internal relay and adds a new header:

Received: from server1.MYPROVIDER.TLD (A.B.C.D)
        by server2.MYPROVIDER.TLD with SMTP; ...

Unfortunately, when GMail fetches the message from MyProvider, it runs the SPF check against the IP address of the last Received header (A.B.C.D), that is the IP address of MyProvider internal relay, and not the one authorized by PayPal for outbound email, resulting in a SPF fail.

Also when messages are not internally relayed by the receiving provider, so when they present only one Received header, GMail fails the SPF check with the error “best guess record for domain of transitioning postmaster@MYOTHERDOMAIN.TLD does not designate as permitted sender“.

From the Common receiver mistakes FAQ of OpenSPF.org…

SPF is designed to work at the border of your network…

… to accept or reject messages as soon as they try to enter, and not after processing, relaying or forwarding have been performed. Moreover it’s intended to be use for SMTP sessions, and not on POP3/fetch.

Email good reputation is hard to achieve; lots of efforts are spent on methods like SPF, DKIM, DMARC to help senders to reach users’ InBox folders and to increase users’ confidence about their content. Surely GMail folks had more than good intentions when they decided to use this policy but, IMHO, a wrong use of these techniques may lead to the opposite results of those intended.

A quick glance at longer than /24 IPv4 prefixes

Distribution of probes which reached the /25 prefix with registered route object

Yesterday RIPE Labs announced a new test to measure propagation of longer than /24 IPv4 prefixes. The scope of these prefixes is to allow small allocations from the 23.128/10 net block that ARIN reserved to facilitate IPv6 deployment, but it could be impaired by filters and routing policies commonly deployed all around the net.

While looking forward to the RIPE results I couldn’t help glancing at the current situation as seen by a bunch of RIPE Atlas probes, so I performed some small tests on my own.

Measurements

The measurements I created are based on 500 random probes from the world wide area and have been used to ping the IP addresses announced by RIPE RIS (AS12654); these IP addresses are split in 2 main groups: one group has 3 addresses each one from a prefix (/24, /25 and /28) with a registered route object, the other group has similar addresses but from prefixes without a registered route object.

Prefix            Pingable IP	 Route object?	My Atlas measurement ID
23.128.24.0/24	  23.128.24.1    yes            1767799
23.128.25.0/25	  23.128.25.1    yes            1767800
23.128.25.240/28  23.128.25.241  yes            1767801
23.128.124.0/24   23.128.124.1   no             1767802
23.128.125.0/25   23.128.125.1   no             1767803
23.128.125.240/28 23.128.125.241 no             1767804

All measurements use the same set of probes, taken from the first measurement (the /24 prefix with registered route object).

Of course, mine is only a partial view (500 probes) taken from a bigger partial view (all the RIPE Atlas probes) of the global Internet; please take my results with regard of this.

Results

Data collected by RIPE Atlas have been parsed with a small python script that I used to match the results. It uses the RIPE Atlas Sagan library.

Reachability

From the first 3 measurements, 497 probes received at least one response from the /24 prefix with the registered route object; of those, only 57 received responses from the /25 prefix and 41 from the /28 prefix too.

Also 497 probes received responses from the /24 prefix without the registered route object; of those, 52 received responses from the /25 prefix and 38 from the /28 prefix too.

Longer than 24 prefixes reachability

Performances

For those probes which reach the /25 and the /28 prefix the average RTT seems to be pretty stable regardless of the prefix size.

Longer than 24 prefixes with route object  RTT avg

Longer than 24 prefixes without route object  RTT avg

Probes distribution

Here it is:

Distribution of probes which reached the /25 prefix with registered route object

Distribution of probes which reached the /25 prefix with registered route object – Click to enlarge

More and deeper tests may be useful to reveal AS paths and peering/transit relationships impacts on these measurements; maybe I’ll blog another post as soon as possible, or – better yet – maybe RIPE Labs folks will write about them in their results.

Conclusions

As far as I can see from the measurements I run (based on a small sample of probes), reachability of smaller than /24 prefixes is almost unaffected by the presence of registered route objects; who receives the registered net also receives the unregistered one. From the performances point of view, when the smaller prefixes are reached, RTTs are not better nor worse than /24 prefixes.

The real question is: what’s the portion of Internet that right now succeed to reach these prefixes?

I guess that some efforts will be needed in order to have ISPs to review their routing policies and accept the new small prefixes from the reserved pool.

References

RIPE Labs, Propagation of Longer-than-/24 IPv4 Prefixes – https://labs.ripe.net/Members/emileaben/propagation-of-longer-than-24-ipv4-prefixes

ARIN, Policy Proposal 2008-5 – Dedicated IPv4 block to facilitate IPv6 deployment- https://www.arin.net/policy/proposals/2008_5.html

ARIN, Number Resource Policy Manual (NRPM), adoption of Dedicated IPv4 block to facilitate IPv6 Deployment – https://www.arin.net/policy/nrpm.html#four10

ARIN, announcement of 23.128.0.0/10 reservation – https://www.arin.net/announcements/2014/20140130.html

RIPE Atlas: a script to show ASes traversed in traceroute

I released a small Python script which reads results from RIPE Atlas traceroute measurements and shows Autonomous Systems traversed by probes to reach the target: ripeatlastracepath.

It uses a library that I wrote to cache RIPEstat results about IP addresses details (ASN, prefix, …), in order to improve performance and to avoid a flood of requests: ipdetailscache.

More details can be found on my GitHub profile page.

A demo can be found here. It can’t be used to process other measurements, it only shows results from measurement ID 1674977, a traceroute from 50 probes all over the world toward www.ripe.net. You can drag&drop ASes to build the layout that best describes your scenario and, once done, you can “save” the graph for later usage. In this demo the “Load graph” button gives a preset of JSON data representing the example graph below:

Graph of traceroute to www.ripe.net

These scripts are not so elegant, but do the job! ;) They are on GitHub.com, feel free to use/edit/fork/improve them as you whish!

Avoid Cisco FIB/TCAM exhaustion on full BGP table feed

The number of IPv4 prefixes in the global BGP table is approaching the limit of many Cisco products, such as 7600/6500 RSP720/Sup720 and some ASR1000, which may hold a maximum of ~500K routes in their FIB (the Forwarding Information Base, where only best paths are stored).

These routers can usually handle a bigger load of prefixes, they can also be used to receive the full BGP table from many upstream providers concurrently, but they can’t manage more than 500/512K entries in their RIB or FIB.

Routing protocols to FIB

On 6500/7600 platforms, the default PFC3BXL/PFC3CXL TCAM configuration provides 512K entries for IPv4 and 256K for IPv6; the show mls cef maximum-routes command allows to show the running TCAM layout, that can be changed by entering mls cef maximum-routes ip … (it requires a reload).

7600(config)#mls cef maximum-routes ip 768
Maximum routes set to 786432. Configuration will be effective on reboot.

7600#show mls cef maximum-routes
FIB TCAM maximum routes :
=======================
Current :-
-------
 IPv4 + MPLS         - 512k (default)
 IPv6 + IP Multicast - 256k (default)

User configured :-
---------------
 IPv4                - 768k
 MPLS                - 16k (default)
 IPv6 + IP Multicast - 120k (default)

Upon reboot :-
-----------
 IPv4 + MPLS         - 512k (default)
 IPv6 + IP Multicast - 256k (default)

While this method allows to patch the immediate requirement for more IPv4 FIB space, it has the side effect of reducing the IPv6 resources, which are supposed to be more and more needed in the future.

Some ASR1000 platforms support up to 500K IPv4 routes and, as far as I know, they don’t allow RIB/FIB repartition.

In some scenarios, in order to continue using these routers while avoiding performance impairments, the BGP – Selective Route Download feature may help.

Routing protocols - table-map - FIB

The BGP—Selective Route Download feature allows a network administrator to selectively download some or none of the BGP routes into the Routing Information Base (RIB). (Cisco.com)

Sample scenario

Suppose to have a border router which receives 3 BGP feeds: some prefixes from a multihomed customer, some from an Internet exchange and the full table from a transit provider. This border router is also connected to a core router, where other border routers and the rest of the network is connected too. Configuring the table-map … filter option on the border router allows to prevent its FIB to be loaded with a lot of useless prefixes, which may be summarized with a default route toward the transit provider.

BGP prefixes filtering with table-map

Filtering may be based on many parameters; the one I used here is the next-hop address of each learned prefix. The final configuration looks like this:

router bgp xxx
 address-family ipv4
  table-map BGP-Table-IPv4 filter
!
route-map BGP-Table-IPv4 permit 10
 match ip next-hop BGP-NextHop-OK
route-map BGP-Table-IPv4 deny 10000
!
ip access-list standard BGP-NextHop-OK
 deny Transit_Provider_IP
 permit any

In order to minimize the changes to the standard configuration, an access-list is created which only denies the transit provider’s next-hop address.
Preferred prefixes learned from the Internet exchange and from the multihomed customer are permitted (not denied) by the access-list and, consequently, by the BGP-Table-IPv4 route-map referenced in the table-map command. Other prefixes are denied by the route-map and prevented from entering into the FIB, but they are anyway kept in the BGP database and announced toward the core router. Local-pref and other features may be configured on the routers to implement usual routing policies. Of course, in this example the core router must have a big enough FIB to handle the full BGP!

Please note that before configuring this feature, specific considerations are needed about local routing policies and traffic engineering mechanisms; special attention should be paid to locally generated prefixes announced to the border router by the rest of the network. As always, do it at your risk!

References

CIDR Report: Summary of total route table size for the past 7 days

Cisco.com: Cisco 7600 Series Route Switch Processor 720 Data Sheet

Cisco.com: Cisco ASR 1000 Series Embedded Services Processors Data Sheet

Cisco.com: BGP – Selective Route Download

RIPE68: Content blocking methods and their impacts

Today, in Warsaw, during the RIPE68 morning session reserved for the Cooperation Working Group, Olaf Kolkman kindly presented my work about Content blocking methods and their impacts.

Olaf’s presentation was the first in a series of 3, all about censorship and censorship circumvention.

Content Blocking Methods And Their Impacts

My work was born within a discussion on the RIPE Co-op Working Group mailing-list with the goal of depicting the impact of various content blocking mechanisms, in order to reach a final guide for policy makers and legislators called to solve social issues by using web filtering methods.

Unfurtunately, the time slot allocated to Olaf’s presentation was too small to allow him to give a full overview of impacts, side effects and hidden costs related to content blocking methods. More details can be found on the online resources:

– on the RIPE68 Cooperation Working Group Agenda you can find the slides and the video of the presentation;

– here on my blog I’m hosting the first version of the document A Technical Overview of Content Blocking Methods;

Download

– on the IETF website you can find the other document that Olaf talked about in his presentation, Technical Considerations for Internet Service Blocking and Filtering by IAB.