Support swap Fn and CtlL keys (how it should be) for the x200, middleton #32

ypid · 2017-06-18T21:18:36Z

For older Thinkpad models like the xx00 series the EC can not be configured to swap Fn and Ctrl. This setting would usually be exposed in the BIOS setup or in nvram when using coreboot. Enabled by default for models where this can not be configured.

Updates: #4
Status: Confirmed working on a x200. My x200 shipped with libreboot so ~~I don’t have the factory rom which would come in handy to flash the EC~~ :) Edit: Got it :)

I will do a bit more work on this: https://notabug.org/libreboot/libreboot/issues/146#issuecomment-5006

hamishcoleman · 2017-06-20T08:05:01Z

One reason why the generated x200 img file might not boot could be the fix-hdd-image-%i.patch files.

There are only two - one for each of two sizes of hdd image, so if the hdd is neither 65536 sectors nor 75776 sectors, there is no patch.

I've just added code to check for the lack of a matching disk image patch on the master branch. If you do have a third disk image size, I probably should bite the bullet and write a tool to fix the partition table.

ypid · 2017-06-20T16:39:59Z

I probably should bite the bullet and write a tool to fix the partition table.

A generic solution is always preferred. Maybe one already exists? Have not checked yet.

ypid · 2017-06-23T19:39:56Z

Lucky you, the old BIOS version used an 24117248 byte image (base for the patch by middleton) but the newer BIOS version uses a supported 38797312 byte image (same EC fw version). Forward porting the patch.

ypid · 2017-06-24T19:49:35Z

Updated. I ported the patch to the latest EC firmware version to benefit from the battery charging fix from it. I confirmed that the ctrl fn swap works using all added patched images on a x200.

Unfortunately, just as I wanted to actually start using the laptop, I bricked it! I must have gotten the timing just right after the dos tool finished and rebooted into the bios (where the EC fw is transfered from SPI flash to EC flash as it seems) when I unplugged the USB DVD rom drive I tested with to confuse the BIOS and interrupt the EC flashing. But that was after I confirmed that patched.x200.iso works. I added a warning to the readme …

hamishcoleman · 2017-06-26T05:50:57Z

Makefile

+
+patch_disable_fn_ctrl_swap:
+	$(call patch_disable,008_fn_ctrl_swap.patch)
+


This method for enabling and disabling patches has gotten unwieldy, especially since there are now multiple non-overlapping hardware types with different sets of patches.

True. One the other hand, three patches are not that much either. If more get added then it should probably be optimized.

hamishcoleman · 2017-06-26T05:51:54Z

Makefile

+	$(info patched.t430.257.112.img - for patching Thinkpad T430 BIOS 2.57/EC 1.12 - no keyboard patch)
+	$(info patched.x200.314.106.img - for patching Thinkpad x200 and x200s BIOS 3.14/EC 1.06 - only Fn Ctrl swap (middleton))
+	$(info patched.x200.322.106.img - for patching Thinkpad x200 and x200s BIOS 3.22/EC 1.06 - only Fn Ctrl swap (middleton))
+	$(info patched.x200.img - for patching Thinkpad x200 and x200s BIOS 3.22/EC 1.07 - only Fn Ctrl swap (ported middleton))


This list is getting a little long to be quickly understandable - perhaps splitting it up in some way would help.. Any thoughts?

Valid point. I gave it a try. The latest version should be enough for list_laptops.

hamishcoleman · 2017-06-26T05:54:46Z

Makefile

+	cp --reflink=auto $< $@
+	./mec-tools/mec_csum_outer -f $@
+	./mec-tools/mec_csum_outer -c $@
+


I dont want to add another new "csum_ok" file type - instead, in the rule where the %.img file is created, it should either fix the checksum or not allowed the file to be created.

Excellent idea! This simplifies this case.

hamishcoleman · 2017-06-26T05:56:10Z

Makefile

+	else \
+	 cp --reflink=auto autoexec.bat.template $@.tmp; \
+	fi
+	sed -e "s%__DIR%`mdir -/ -b -i $<@@$(FAT_OFFSET) |grep FL2 |cut -d/ -f3 | head -n 1`%; s%__FL2%`mdir -/ -b -i $<@@$(FAT_OFFSET) |grep FL2 |cut -d/ -f4 | head -n 1`%; s%__DESC%`scripts/describe $<`%; s/__BUILDINFO/$(BUILDINFO)/" --in-place $@.tmp


This is an ugly special case check. See also below where I talk about autoexec.bat.command_com.template

hamishcoleman · 2017-06-26T05:57:57Z

Makefile

+mec-tools/mec_csum_outer: mec-tools/Makefile
+	git submodule update
+	make -C mec-tools
+


This is an exact repeat of the mec-tools/mec_encrypt - instead of duplicating it, there should be a known "tag file" in the mec-tools dir that only exists after a compile, and all rule to watch the mec-tools contents and build that tag file. Then all rules that need mec-tools will depend on that builder rule.

Good catch, fixed.

hamishcoleman · 2017-06-26T06:03:19Z

autoexec.bat.command_com.template

+echo.
+pause
+echo on
+command.com


Can you expand a little on what this is actually doing? Have Lenovo built a custom command.com? If so, what is interpreting this batch script?

Not a batch script but some binary blob which was invoked exactly the same by the original autoexec.bat. I simply copied last line from the original file.

Looking closer, it seems that /FLASH/COMMAND.COM is a compiled script which calls /FLASH/UPDTFLSH.EXE - another compiled script - which in turn calls FLASH/PHLASH16.EXE - and it looks like that one has some commandline options for specifying which files to flash.

I also wonder if the default flash scripts will allow you to re-flash a "same version" image - the xx30 tools certainly didnt, which is why zmatt worked out the dosflash commandline options.

Either way, to clean this up, it looks like some thought will be needed on how to make the autoexec bat generation better in the future.

I also wonder if the default flash scripts will allow you to re-flash a "same version" image

This seems not to be an issue here. The EC firmware could just be downgraded with the patched.x200.314.106 image/flasher. I can confirm that this worked. I also tried to increase the version numbers but this was not reflected in the BIOS message. To be continued.

PHLASH16.EXE

This is worth a try, thanks!

hamishcoleman · 2017-06-26T06:04:12Z

autoexec.bat.template

@@ -4,7 +4,8 @@ cd c:\flash
 cls
 echo.
 echo.
-echo This will flash your firmware to support a classic 7-row keyboard
+echo This will flash your EC firmware to support a classic 7-row keyboard (default)
+echo or other modifications as configured at modification time.


The build system should probably keep track of which patches are enabled and applied and insert that here, to be 100% clear and correct

hamishcoleman · 2017-06-26T06:04:54Z

docs/CONFIG.txt

+  For older Thinkpad models like the xx00 series the EC can not be configured
+  to swap Fn and Ctrl. This setting would usually be exposed in the BIOS setup
+  or in nvram when using coreboot.
+  Enabled by default for models where this can not be configured.


Possibly need to make it clear which patches are available for which types of laptop

hamishcoleman · 2017-06-26T06:07:36Z

scripts/checksum


    if [ "$RM_ON_FAIL" ]; then
-        rm $FILE
+        rm "$FILE"
    fi

    exit 1


all these "fixes" patches should go into a separate patch and PR (similarly with some of the copyFL2 stuff below)

True, I started with this one and moved it out of this PR and into #34. I will address your other comments in a few days :)

hamishcoleman · 2017-06-26T06:11:07Z

scripts/copyFL2

+                exit 1
+            fi
+        done
+    fi


This is all way too complex. I've been thinking about it and now think that the caller should simply provide an arg that is the pattern to match. It could default to "FL2", but for the x200, it could be told to use "01B9x00.FL2" as the pattern, then the same "more than one match" logic could apply.

It just works already. Your proposal would require more work for people adding such images with more than one FL2 file. Can‘t we just stay with this? I tried to validate all cases so that it should not break something.

It doesnt require any additional work - there is already an assumption that you have the correct linux filename for the FL2 file (though, I see you have just used 'x' in that filename)

Also, I have already found ISO files from Lenovo where they have included multiple, unrelated FL2 files.

Finally, I'm trying to move towards a state where none of the inner names are hidden - and all are visible in the Makefile - which would then allow me to build an auto dependency maker and remove those hardcoded lists from the Makefile. So adding this as a param actually makes work simpler in the future.

I have added the pattern matching feature described here to the master branch as I hope that might make it clearer.

hamishcoleman · 2017-06-26T06:17:04Z

Ouch! thats good to know about plugging things in during flashing, but sorry that you had to brick something to learn it.

There looked to be a Russian website that had EC flash-programmers, so you might find that there is a way to attach an in system programmer.

Do you have schematics for the x200? I had a look at my x220 schematics and it looks like most of the JTAG pins were easily accessible (all but the CLK and RST lines are wired out to the KBDID$n pins on the x220)

For older Thinkpad models like the xx00 series the EC can not be configured to swap Fn and Ctrl. This setting would usually be exposed in the BIOS setup or in nvram when using coreboot. Enabled by default for models where this can not be configured. Updates: hamishcoleman#4

The outer checksum has been observed on all Lenovo EC firmware files analyzed so far. This avoids the special case for the x200 which only has the outer checksum.

Updates: hamishcoleman#32

…ould be worked on

ypid · 2017-06-29T21:18:36Z

Yes, the manual mentions JTAG and also a on-board programming mode so it should be possible. And yes, I got lucky and have all the schematics :-) I already had a detailed look at the board with OpenBoardView. I will get back to this PR when I have tested it again. To speed this up, I ordered a second MB.

ypid · 2017-07-03T22:27:03Z

Nice work on the master branch! I will need to rebase this PR later :)

I tested with a second MB and bricked it again. Seems I am becoming good at this :) One thing I noticed is that the dos flashing tool checks the EC checksum and refuses to do anything when it is not correct.

From BIOS 3.21/EC 1.06 → BIOS 3.14/EC 1.06

Did not reflash the EC

From BIOS 3.14/EC 1.06 → BIOS 3.22/EC 1.06

Warning mentions to either eject CD before reboot or after reboot finished, but not in between. Earlier versions did not mention this which is why I bricked my first one :)
Did not reflash the EC

From BIOS 3.22/EC 1.06 → BIOS 3.14/EC 1.06 (increased EC version to 7XHT24WX)

Did not reflash the EC

From BIOS 3.14/EC 1.06 → BIOS 3.14/EC 1.09 7XHT26WW (manually increased EC version)

Did reflash the EC
Bricked the EC

@@ -32,9 +32,9 @@
 000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 00000200  43 4f 52 50 2e 20 49 42  4d 20 32 30 30 31 20 41  |CORP. IBM 2001 A|
 00000210  4c 4c 20 52 49 47 48 54  53 20 52 45 53 45 52 56  |LL RIGHTS RESERV|
-00000220  45 44 00 00 00 00 00 00  10 60 b9 00 13 74 62 b4  |ED.......`...tb.|
+00000220  45 44 00 00 00 00 00 00  10 90 b9 00 13 74 62 b4  |ED...........tb.|
 00000230  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
-00000240  37 58 48 54 32 34 57 57  5f 5f 5f 5f 5f 5f 5f 5f  |7XHT24WW________|
+00000240  37 58 48 54 32 36 57 57  5f 5f 5f 5f 5f 5f 5f 5f  |7XHT26WW________|
 00000250  03 06 03 10 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 00000260  00 00 00 00 28 43 29 20  43 6f 70 79 72 69 67 68  |....(C) Copyrigh|
 00000270  74 20 49 42 4d 20 43 6f  72 70 2e 20 32 30 30 31  |t IBM Corp. 2001|

hamishcoleman · 2017-07-04T02:08:56Z

Now it sounds like you need to start practising hardware flashing :-(

It is good that the flash tools check the checksum - that at least means that we know we are not accidentally flashing bad data. Did you do any tests with the phlash16.exe tool?

I know I made big changes to the master - effectively forcing a rebase, but what we have now is pretty close to what was in my head. It should allow supporting multiple different major types of hardware in a straight-forward way. There are still a bunch of things I can see to change, but I think it is time for me to go back to waiting until the feature is needed before ripping the infrastructure apart

ypid · 2017-07-04T05:26:50Z

Now it sounds like you need to start practising hardware flashing :-(

Sooner or later that will come in handy. Will probably require a bit of SMD soldering to change the mode of the EC and a second read of the data sheet.

Did you do any tests with the phlash16.exe tool?

I gave it a quick try but got "Cannot flash when Memory Manager (e.g. HIMEM) are present".

ypid · 2017-08-03T17:09:57Z

Just a little update: I still intend to finish this project, rebase the PR and work on getting it merged. But currently I don’t find much time. Maybe in a couple of months. Two more working motherboards are waiting to be bricked (really cheap on ebay) and one day hopefully all of them can be un-bricked and put into service :)

hamishcoleman · 2017-08-04T00:18:13Z

Thanks for the update - I've been busy and away myself, so have not had much time to spare either.

I might have some time before you do, so - if you dont have any objections - I might be able to rebase and start merging sooner than you (with appropriate warnings about bricking, it seems :-(

ypid · 2017-08-04T17:03:44Z

Thanks! No objections from my side. About the bricking: It seems like the M$ Windows tool might be more resilient here. middleton provided patches for dozens of IBM/Lenovo models and users reported basically no issues. Anyway, it has to work without Windows, especially for the X200 :)

herbsmn · 2018-08-01T23:29:39Z

I have an x200 with libreboot on it and would love to liberate the EC. It sounds like work is still being done to make this process stable though. Is that correct?

ypid · 2018-08-02T11:29:08Z

Not right now unfortunately as far as I know. I linked to a few other places. Also swiftgeek is interested in this and might be able to help you. Feel free to pick up where we left off.

herbsmn · 2018-08-05T03:08:18Z

So does this support any libreboot supported laptops? I also asked this question over here: lynxis/frenc#1

ypid · 2018-08-06T06:18:12Z

This particular PR you asked in does. I successfully tested it on Libreboot until I bricked it. As I see it, the system firmware and the EC firmware are two separate things which don't interact that much, other than system firmware updating the EC firmware (probably not supported by Libreboot yet) and configuration of the EC from the system firmware setup (Supported by Libreboot). About frenc: This is very hardware dependent. Different Thinkpads use different ECs. If you are lucky, similar models might share the same EC. Please check the docs of this project where some details are documented and learn the background. What you want requires work that does not do itself/by other people. You can either commit to it and spend a lot of time or we don't have a free EC firmware for the xx00 until somebody else commits to do it.

herbsmn · 2018-08-20T00:10:49Z

@ypid: What model of libreboot laptop were you using with @lynxis' liberated h8s EC? How well did it work? Did the liberated EC cause the laptop to get bricked? Also, how did you flash it on the laptop? The instructions aren't very well developed.

ypid · 2018-08-20T21:52:30Z

I can not help you much here. I only flashed the stock/factory EC firmware (slightly patched) using the update mechanism which is supposed to be used by end users (done as part of system firmware update). I have not played with the EC firmware rewrite but I am aware of it and would appreciate a free EC firmware.

herbsmn · 2018-08-20T23:08:13Z

@ypid Thanks for the response. I misread your last comment. I too would appreciate a free EC firmware.

ypid force-pushed the add-x200 branch 2 times, most recently from 8b835c6 to ea0f296 Compare June 24, 2017 19:39

ypid changed the title ~~[WIP] Support swap Fn and CtlL keys (how it should be) for the x200, middleton~~ Support swap Fn and CtlL keys (how it should be) for the x200, middleton Jun 24, 2017

ypid force-pushed the add-x200 branch from ea0f296 to fb0aeb8 Compare June 24, 2017 19:50

hamishcoleman requested changes Jun 26, 2017

View reviewed changes

ypid force-pushed the add-x200 branch from 0e9a317 to aedd6cb Compare June 26, 2017 15:27

ypid added 5 commits June 26, 2017 19:17

Split make list_laptops into make list_laptops_all_versions

1abface

Ensure the outer EC fw checksum is always fixed after patching

e738afd

The outer checksum has been observed on all Lenovo EC firmware files analyzed so far. This avoids the special case for the x200 which only has the outer checksum.

Use "tag file" for mec-tools build to avoid redundant makefile targets

ac64346

Remove *.img.diff in make clean. Was missing before.

4e43dd5

ypid added a commit to ypid/thinkpad-ec that referenced this pull request Jun 26, 2017

Fix shell script warnings reported by shellcheck for scripts/checksum

7304988

Updates: hamishcoleman#32

ypid mentioned this pull request Jun 26, 2017

Fix shell script warnings reported by shellcheck for scripts/checksum #34

Merged

ypid force-pushed the add-x200 branch from e4995dd to 4e43dd5 Compare June 26, 2017 22:20

ypid referenced this pull request Jun 29, 2017

Allow the makefile to explicitly specify to copyFL2 which FL2 file sh…

4350623

…ould be worked on

Document EC chip used in a x200

93dacac

ypid mentioned this pull request Aug 6, 2018

Does this work on any Libreboot supported ThinkPads? lynxis/frenc#1

Open


		patch_disable_fn_ctrl_swap:
		$(call patch_disable,008_fn_ctrl_swap.patch)

Support swap Fn and CtlL keys (how it should be) for the x200, middleton #32

Are you sure you want to change the base?

Support swap Fn and CtlL keys (how it should be) for the x200, middleton #32

Conversation

ypid commented Jun 18, 2017 • edited

hamishcoleman commented Jun 20, 2017

ypid commented Jun 20, 2017

ypid commented Jun 23, 2017

ypid commented Jun 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hamishcoleman commented Jun 26, 2017

ypid commented Jun 29, 2017

ypid commented Jul 3, 2017 • edited

From BIOS 3.21/EC 1.06 → BIOS 3.14/EC 1.06

From BIOS 3.14/EC 1.06 → BIOS 3.22/EC 1.06

From BIOS 3.22/EC 1.06 → BIOS 3.14/EC 1.06 (increased EC version to 7XHT24WX)

From BIOS 3.14/EC 1.06 → BIOS 3.14/EC 1.09 7XHT26WW (manually increased EC version)

hamishcoleman commented Jul 4, 2017

ypid commented Jul 4, 2017

ypid commented Aug 3, 2017 • edited

hamishcoleman commented Aug 4, 2017

ypid commented Aug 4, 2017 • edited

herbsmn commented Aug 1, 2018

ypid commented Aug 2, 2018

herbsmn commented Aug 5, 2018

ypid commented Aug 6, 2018 • edited

herbsmn commented Aug 20, 2018 • edited

ypid commented Aug 20, 2018 • edited

herbsmn commented Aug 20, 2018

ypid commented Jun 18, 2017 •

edited

ypid commented Jul 3, 2017 •

edited

ypid commented Aug 3, 2017 •

edited

ypid commented Aug 4, 2017 •

edited

ypid commented Aug 6, 2018 •

edited

herbsmn commented Aug 20, 2018 •

edited

ypid commented Aug 20, 2018 •

edited