It seems MSVC has a problem with Eigen's massive use of inlining. Some tests which take minutes to compile with GCC/clang take up to an hour to compile with MSVC 2017 as reported here: https://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2019/06/threads.html.
The timings for MSVC 2019 are better, but not great.
I have set up some test scripts here: https://github.com/bjornpiltz/eigen_compilation_time
To recreate the tables, compile Eigen3.sln in release mode from within Visual Studio and process the build log with eigen_compilation_time_msvc.Rmd.
I would suggest creating a minimal example, which can be reported to Microsoft as this is obviously an issue on their side.
The link to the mailing list should be https://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2019/06/msg00010.html
Here are the timings:
And for reference GCC 7: