Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Legacy Makefiles #216

Draft
wants to merge 36 commits into
base: master
Choose a base branch
from
Draft

[WIP] Legacy Makefiles #216

wants to merge 36 commits into from

Conversation

Neui
Copy link
Contributor

@Neui Neui commented Feb 17, 2019

Includes an makefile generator that should be able to generate non-independent makefiles with certain settings (system, compiler). Builds upon PR #184 (so that should be merged first), originally from PR #164. Still Work In Progress.

Unlike the makefiles in the historical folder, this creates one big makefile that contains everything you need to build. It also uses the substitution feature heavily to cut down the size by not listing each file and how to commpile.

At the moment it generates the following makefiles:

  • posix (should be compatible with basic POSIX systems, including linux/gcc)
  • gcc (like POSIX but with gcc stuff)
  • mingw (like GCC but for windows, currently untested)
  • dos_gcc (like gcc/mingw but uses a dos-specific option -m486, untested)
  • win32cl (for Microsofts CL compiler, usually included with Visual Studio, untested)
  • amiga (uses SAS/C and smake, untested)
  • amiga_gcc (uses gcc but also uses amiga system tools, untested)

Note that most are actually untested. Also it only builds, it doesn't do any testing or similar, since that would currently overcomplicate things.

I've also noticed that you can't really built inside of (Free)DOS because of the 8.3 name limitation with undistinguishable names (like decode_6502.c and decode_65816.c would both be truncated to decode_6.c). DOSBOX seems to use (Windows-Style?) short names, so that might work, but untested because of OpenWatcom.

This PR currently includes OpenWatcom as a compiler, BUT their C compiler for include files first searches the current working directory instead of the directory of the file currently being compiled, unlike other C compilers like GCC or CLang. (I am not sure if SAS/C (amiga) also has different behaviour.) Because of this, I probably gonna put it into a separate branch and maybe work on it on a later date (or maybe never). In other words, this PR won't have OpenWatcom support at the time of the merge.

A question I have is should we put the generated files also in the repo or offer an separate source package with such files pre-generated? I would prefer the last one since the git repository shouldn't contain any generated files (since they can be generated and thus be "outdated").

(Note that GitHub displays commits in date order. I heavily rebased this and since I didn't fixed the dates, this appears nonsensical. You might want to view this in a client that orders the commits on how it is actually stored, like most git clients).

@cr1901
Copy link
Collaborator

cr1901 commented Feb 17, 2019

I've also noticed that you can't really built inside of (Free)DOS because of the 8.3 name limitation with undistinguishable names (like decode_6502.c and decode_65816.c would both be truncated to decode_6.c).

Some DOSes have LFN support via a driver, and Idk whether Ville ever actually built WLA on DOS (@vhelin could you chime in :D?). I just know that DOS machines w/ 386 CPU and above had binaries for WLA for a while.

Main reason to keep DOS support around IMO is that there are decent tools for retro dev which only still exist on retro systems, so might as well keep the assembler around as well :P.

In other words, this PR won't have OpenWatcom support at the time of the merge.

Fortunately, the legacy Makefiles never had this either :) (at least I don't think they did- if they did, it was because I added it, and I never meaningfully used that support before CMake.)

@vhelin
Copy link
Owner

vhelin commented Feb 17, 2019 via email

@Neui Neui force-pushed the legacy-mkfiles branch 2 times, most recently from cfd0737 to 6d010a4 Compare February 17, 2019 19:12
@Neui
Copy link
Contributor Author

Neui commented Feb 17, 2019

Yeah! If we can have a working (pure) DOS support, that would be nice.

Note that I haven't tested compiling ON (Free)DOS yet. It also uses an option called -m486, which probably means it would compile for i486, where DOS also supports i386, so it might not work for the full range of hardware.

Also there are of course cross compilers like DJGPP-GCC, that might work with the CMake build system generator. I haven't tried it yet.

PS. If the generated Makefile(s) are in a separate source package, not in the Git repo, where would they be? And how to make sure they are in sync with the Git repo?

I mean when a release is being done (doing an annotated tag), then you can upload additional files to that tag(/Release in GitHub) and you could then upload the source code but with those generated files included (which are located in mk which then the user picks out).

PPS. Is this going to replace CMake?

No, they co-exist. CMake can generate files for IDEs (like Visual Studio) and other build systems (Ninja), and better integration with other tools (compilation database, there's also an daemon version).

PPPS. That's a massive amount of work you've done here! :)

Thanks! I actually had this for a while, with good weather now I added amiga support and refined it a little.

(Note that I noticed some mistakes and fixed them, but it still needs testing)

@Neui Neui mentioned this pull request Feb 17, 2019
7 tasks
@cr1901
Copy link
Collaborator

cr1901 commented Feb 18, 2019

I would like to test on Amiga, but I don't have the hardware (perhaps I could set an emulator up if someone can provide me w/ the relevant C compiler, or @Neui can do it and just see if the tests run :)?).

AIUI, one killer app for Amiga for retro dev is their version of Deluxe Paint, which Idk if it has a good equivalent even today!

For VB dev, I can get Paint dot net to generate output files in the appropriate format, but it isn't exactly wieldy (requires python scripts).

@Neui
Copy link
Contributor Author

Neui commented Feb 18, 2019

Note that the "testing" here is just checking whenever it builds and the binaries run.

I can check mingw and win32cl by myself, it's just I haven't done it yet. With dos_gcc I've looked up that there's an LFN program/driver for FreeDOS that I might try (together with DJGPP-GCC). Since I never had a amiga and it seems that (at least) the OS is not free, and I don't really want to give out money for something I then probably never use.

Note that my main procedure was to just look at the makefiles and try to recreate the commands.

@Neui Neui force-pushed the legacy-mkfiles branch 7 times, most recently from c271793 to b81874b Compare February 19, 2019 17:41
@Neui
Copy link
Contributor Author

Neui commented Feb 20, 2019

Current state of the art:

Using DJGPP-GCC I am able to compile on FreeDOS (on a virtual machine) and it works. Note that it seems it compiles (and runs) programs in protected mode, not real mode, so it isn't completely DOS compatible (or at least pre 286/386 CPUs). Currently the main "problem" is that for cleanup, it complains that the command is too long. I just have to implement wildcard expression and it should be OK. In addition, I've renamed dos_gcc.mk to dosdjgpp.mk, because it was made and tested with DJGPP-GCC.

Microsofts CL compiler (win32cl.mk) also works. I've removed /Zw since VS2013 and VS2017 doesn't detect it. MinGW also works.

I also plan to polish the mk/README.md, mainly adding make variables you can change. Feedback appreciated.

There is still no testing about the makefiles for Amiga, since I don't have one. If nobody can test it, what about excluding them until we can? Since otherwise this PR (and the CMake one) would then just get stalled for no real added benefit if it doesn't work and most people probably gonna compile on modern systems anyway; It can always be added later.

Also it seems some old makefiles also strip the binaries and some don't. Should I also include stripping for all binaries? I'd say yes to get the size down.

@cr1901
Copy link
Collaborator

cr1901 commented Feb 21, 2019

@vhelin When this is complete, perhaps one of us could look into getting CI running to make sure each target builds (and tests pass)?

Using shell scripts specifically for CI purposes should be reasonable. Just need to figure out how to reliably test Amiga target via emulation (DOSBOX already has a mechanism to run a single binary and return stdin/stdout to the OS).

@Neui
Copy link
Contributor Author

Neui commented Feb 21, 2019

In the old PR I actually had some CI support for some services, but not really for this legacy makefiles. I'll have to look up about DOSBOX. Also, since you can/need to buy the amiga OS, emulation becomes hard because the build scripts are globally readable and if they contain how to get that OS for free, it might be bad for us.

@vhelin
Copy link
Owner

vhelin commented Feb 21, 2019 via email

@vhelin
Copy link
Owner

vhelin commented Feb 21, 2019 via email

@vhelin
Copy link
Owner

vhelin commented Feb 21, 2019 via email

@cr1901
Copy link
Collaborator

cr1901 commented Feb 21, 2019

I have to admit that I don't know completely what I would need to do. :)

I have experience w/ Travis, I could look into it.

If no-one else has the time or energy or resources to test the makefiles for Amiga, I think I could setup WinUAE with SAS/C C-compiler...

That would be wonderful :). How would you get around the copyright problem for the ROMs that @Neui mentioned? There seems to be an alternative OS called AROS that can run Amiga 3.x binaries.

Amiga 4.x can run GCC; dunno offhand how to build a libc for that, but even ancient versions of GCC (like 2.x) shouldn't choke on wla-dx.

@vhelin
Copy link
Owner

vhelin commented Feb 21, 2019 via email

@vhelin
Copy link
Owner

vhelin commented Feb 22, 2019 via email

@vhelin
Copy link
Owner

vhelin commented Feb 22, 2019 via email

@cr1901
Copy link
Collaborator

cr1901 commented Feb 22, 2019

It's really exciting for me too. I'm glad you were receptive to keeping the source c89 compatible. I'm glad its paying off now :3.

If ppl get worried about buffer overflows later (and I agree that some c89 functions are dangerous), we can provide our own copies of snprintf and friends.

Did your friend give any ideas about how to add Amiga building to CI using emulation (without copyright problems)?

@Neui Neui force-pushed the legacy-mkfiles branch 3 times, most recently from 175f94d to a2fcd04 Compare February 23, 2019 16:56
@Neui
Copy link
Contributor Author

Neui commented Feb 24, 2019

I've implemented RM glob, so deleting on DOS should work. Also, DOSBOX doesn't work since while it converts long filenames to short ones, it actually doesn't support the LFN API, so it actually fails (to find include_file.h). Also it seems you can't really output the console to stdout, and it needs an X display (runs via SDL). While there's an LFN patch for DOSBOX, they just provide "changed" files without an revision on what it is based on. Maybe I'll look into it, but not now.

I didn't implemented stripping because I didn't wanted to create a another variable just for it. I'd say just modify LDFLAGS if you like to strip (or run the strip binary).

I've also noticed the amiga fixes. However I've a few questions: Did you test with this PR? Does the include files depend on where the source file is, or where the call to the compiler is emitted or ..? Does it support relative file paths? Is the / the root directory of an drive or the source folder?

Since it was there before and it seems `-ansi` is much longer there
than `-std=c89` (and `-std=c90`, which was introduced in GCC 4.5).

CMake by default uses the `-std`-parameter, so by emptying the
`CMAKE_C_STANDARD` it should go to the default behaviour.
"legacy" as in it doesn't use CMake. It simply generates makefiles
out of a configuration in `generate.sh`.
There are make implementations that don't like .POSIX, such as
openwatcoms WMAKE.
Windows (and DOS) likes to use a slash `/` for options. However,
they are also used as directory seperators in unix-like systems. So
this adds a new option to convert those slashes in source and binary
directories backslashes.
Special thing when you need to add a pre- and suffix at the source
file listing. Note that the seperator can't be configured yet.

Primarly made for openwatcom linker WLINK.
Allows customization of the CC and LD commands for better
customizability of different commands.
Windows uses for example `.exe`
A file to append to the end of a makefile. Useful to, for example, add
more targets and variables.
It seems the last command might not return 0 and thus the `mk/generate`
script thinks it actually failed when it wasn't.
On Windows CMD (and NMake) you usually execute executables in the
current directory by typing their name. However, on Unix (and
PowerShell) you have to prefix it with `./`.

This change should make it possible to customize that aspect.
@Neui
Copy link
Contributor Author

Neui commented Feb 24, 2019

I've rebased it and pushed it;

git merge --abort
git fetch origin pull/216/head
git checkout FETCH_HEAD

or

git merge --abort
git remote add neui https://github.com/Neui/wla-dx
git fetch neui
git checkout neui/legacy-mkfiles

@vhelin
Copy link
Owner

vhelin commented Feb 24, 2019

bandicam 2019-02-25 01-29-09-403-2

This is what I get...

@Neui
Copy link
Contributor Author

Neui commented Feb 25, 2019

It looks like SMake doesn't support substituting variables, which is a major pain. It also explains why every SMakefile explicitly lists all files and their object files rather than substituting. Maybe I'll do an separate script that is specifically tailored to SMake/SAS.

@cr1901
Copy link
Collaborator

cr1901 commented Feb 25, 2019

It looks like SMake doesn't support substituting variables, which is a major pain. It also explains why every SMakefile explicitly lists all files and their object files rather than substituting. Maybe I'll do an separate script that is specifically tailored to SMake/SAS.

This is probably for the best, unfortunately. There is in practice no such thing as a portable Makefile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants