I’ve been having a problem for a while now: my Android Ice Cream Sandwich (ICS) devices were unable to connect to my L2TP/IPSec PSK VPN. This happened on both my Asus Transformer Prime (TF201) running ICS 4.0.3 and my shiny new Samsung Galaxy S3 running ICS 4.0.4
In order to do my job, I need to connect to my VPN in order to have a secure connection to the LAN, internal servers and my desktop computer. So this problem was major. Yesterday night, I finally took a few minutes to diagnose the problem.
My VPN server is running on Ubuntu 10.04 LTS and is setup using OpenSwan (l2tpd & pluto ipsec). I checked the /var/log/pluto.log while trying to connect my Android device to the VPN, and noticed these errors:
"L2TP-PSK-NAT"[16] 11.22.33.44#15: byte 7 of ISAKMP NAT-OA Payload must be zero, but is not
"L2TP-PSK-NAT"[16] 11.22.33.44#15: malformed payload in packet
| payload malformed after IV
| 7c 98 58 d1 bd 64 bd 43 6f c3 5d 7c 19 e3 23 ef
"L2TP-PSK-NAT"[16] 11.22.33.44#15: sending notification PAYLOAD_MALFORMED to 11.22.33.44:4500
After a quick Google Search on “ics l2tp ipsec vpn malformed payload in packet”, I found the following bug report on Google Code: Issue 23124: Can’t connect to VPN (nexus s – ice cream sandwich).
Comment 203 from strawman is particularly interesting as the person identified the bug and provides a fix. Basically, Google is not following the RFC by the book and OpenSwan doesn’t tolerate this error. The user fixed the building of the ISAKMP NAT-OA packet (byte 7) in /system/bin/racoon and uploaded his patch.
This involves copying the “racoon” binary onto your Android system, so you will need root to do so:
- Download the new “racoon” binary from Comment 203 in the Google Code link above
- Copy the “racoon” binary onto your Android device’s Internal SD-Card (I will assume this is /mnt/sdcard/ in these instructions)
- Open a Terminal or an ADB Shell session
- Get superuser (root) access:
su - Mount the /system filesystem as RW, you need root for this:
mount -o remount,rw /system - Make a backup of the existing “racoon” binary:
cd /system/bin
cp racoon racoon.original - Copy the new “racoon” in /system/bin:
cp /mnt/sdcard/racoon . - Set the proper permissions on the new “racoon”:
chmod 755 racoon
chgrp shell racoon - Remount the /system filesystem as RO:
mount -o remount,ro /system
That’s it, you are all done. Now, try connecting to your L2TP/IPSec PSK VPN with your Android device again, and within a few seconds, you should finally be able to connect successfully.
In case something goes wrong, the patch doesn’t work, or if a future OTA upgrade fails, just remember to delete the “new” racoon and rename the original one back to its proper name.
P.S.: All credit goes to strawman on Comment 203. The new “racoon” binary can be downloaded from the Google Code page linked above.
Thank you for this valuable information. It solved my connection problems!
Dear All,
I tried implementing above patch on my Android MID- Mtk6575 Chinese Tablet.
Using ADB or terminal I was unable to chgrep hence i used root explorer to do so but still it is giving message unsuccessful.
I am connecting to Checkpoint VPN.
the error on the Firewall is as below:-
Information IKE: Main Mode Failed to match proposal:
Transform: AES-128, MD5, Pre-shared secret, Group 2 (1024 bit)
Reason: unsupported authentication method 65001
any help that can resolve this issue is highly appreciated, i am stuck with this from almost a week.
Best Regards
Sarfaraz
Hi, thanks for this tutorial! I’m trying to solve the same problem. I unlocked the bootloader and rooted my phone (Xperia Arc S Android 4.0.4). In step 4 of your tutorial, after su, it asks for a password. But which password do I have to enter there?
Found it. I tried to execute the code in Terminal on my Mac, while I should have done it in some terminal app on my Android phone… I’m there now. But now the copy command (cp) is not working. Now let’s find a solution for that. Thanks anyway.
Found it (part 2)!
I had to write down the word ‘busybox’ in front of the commands ‘cp’, ‘chmod’ and ‘chgrp’ in order to let the script work. And ‘shell’ had to be ‘2000’ in my case. It’s the group-id instead of group-name.
But unfortunately without luck, I still don’t get a VPN connection. In fact, I only get a time-out. It doesn’t even try to connect. The original racoon at least tried to connect, but ended with a ‘failed’ message or so. So now I am thinking I should change another parameter somewhere, but where.
I found some differences between your solution and the original #203 solution. You say chmod 755 while #203 says chmod 777. And you say chpgrp, while #203 doesn’t mention that at all. I am wondering what made you bring up these changes? What do they depend on? And could it be that I should use other figures than 755/777 or other commands?
Hi Jim,
I believe the reasons for the chmod/chgrp differences between post 203, my own post, and possibly some other people’s, is the way the original raccoon’s permissions were setup.
So, you are bringing a good point here. It may be a good idea to look at the original racoon’s permission and ownership, and set the new raccoon’s exactly as the old raccoon.
Try a “
ls -l racoon*” to view the permissions of the old file. Let me know and I’ll see if I can help some more.That’s a good one. I should have looked at those properties before overwriting the file, but I haven’t. So I’m afraid I can’t tell you. I did copy the original racoon file to another location, but I suppose the group-id and permissions are gone then.
I did the ls-command several times and during one of them I noticed that the new racoon file was much smaller in size than the original one. Probably something went wrong during copying. So I copied the new racoon file again to the /system/bin and now it’s fine. That is…. at least I don’t get time-outs anymore. But still I don’t get a connection. It simply fails just like the old racoon file did. Using chmod 755 or 777 makes no difference. At this moment I have given up. The trick won’t work for me I’m afraid.
Thanks.
It makes complete sense google doesnt want vpn etc on their phones. harder to track you
I’ve had no luck at all connecting in “Main Mode”. “Aggressive Mode” gets past phase I of the connection process, but then fails. You might want to try “Aggressive Mode”.