Fix potential bug in Stack failing to properly isolate pkg-db environment

[hvr]

Recently a Stack user complained about bugs in Stack in a public venue which as you'll surely agree isn't a suitable place to complain about bugs in Stack and even less so for getting said bugs resolved. Therefore I'm quoting their lament here in Stack's bug tracker where it actually belongs so we can quickly resolve this to everyone's benefit:

I recently decided to start a project using Cabal instead of Stack because I hadn't done that in a while. After a short while I decided to also build it with Stack. I immediately ran into a problem because of the .ghc.environment file. I only managed to rescue myself because I happened to hear about this issue before. I was not happy about the whole situation.

I don't use Stack myself nor do I remember having heard any such bug reports about Stack since this feature was added to GHC 8.0 three years ago. I also reached out to a Stack user who wasn't able to reproduce this. So even though the reported faulty Stack behaviour appears unlikely to me (given there's already four major GHC series out there with this new exciting feature and this is the first time I'm aware of a report of it suggesting a bug in Stack's implementation) I'd like to give the reported malfunction in Stack the benefit of the doubt and assume it being made in good faith, and consequently take it seriously enough to bring it to your attention.

Please let me know if you have any questions about the semantics/mechanics as well as current limitations of this powerful package environment file feature in GHC as there appears to be a lot of not-well-informed misconceptions being dispersed about it. I've been involved in supporting early adopters as well as improving its implementation in response to the technical feedback we received over time, so I most likely can answer most of the question you might have even if the documentation might be lacking and this should help to quickly address this potential bug in Stack in a most effective way.

[dbaynard]

Thanks for raising this, @hvr.

even if the documentation might be lacking

I'm unfamiliar with this feature, myself — would this be the place to start?

With that, I have some questions about the technical details of the problem you report. It would be helpful to have log information, from somebody able to reproduce this issue. Would you please invite the person who raised the issue to comment here, if possible?

[tfausak]

Hi! I'm the user that @hvr is talking about here.

I originally made my comment on Reddit. You can find it here: https://np.reddit.com/r/haskell/comments/b9scx2/package_environment_files_run_counter_to/ek6u69t/

I did not open a bug report for this problem for two reasons: One, it wasn't clear to me which component was at fault, if any (Stack, Cabal, or GHC). Two, I didn't record or remember the exact actions that got me into and subsequently out of this situation.

Since my hand has more or less been forced here, I'm working on reproducing this bug.

[tfausak]

I managed to reproduce this. For context, here is my system information:

$ uname -a
Linux adele 4.18.0-17-generic #18-Ubuntu SMP Wed Mar 13 14:34:40 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ stack --version
Version 1.9.3, Git revision 40cf7b37526b86d1676da82167ea8758a854953b (6211 commits) x86_64 hpack-0.31.1

$ stack exec -- cabal --version
cabal-install version 2.4.1.0
compiled using version 2.4.1.0 of the Cabal library

To get started, set up an empty Stack project somewhere. The location and the resolver don't matter.

$ mkdir /tmp/example
$ cd /tmp/example
$ echo '{ "resolver": "lts-13.0" }' > stack.yaml

Then create a minimal package file. The name and version don't matter.

$ echo 'cabal-version: 2.4' > example.cabal
$ echo 'name: example' >> example.cabal
$ echo 'version: 0' >> example.cabal
$ echo 'library { default-language: Haskell2010 }' >> example.cabal

Then prove that getting a REPL through Stack works:

$ stack build
example-0: configure (lib)
Configuring example-0...
example-0: build (lib)
Preprocessing library for example-0..
Building library for example-0..
example-0: copy/register
Installing library in /tmp/example/.stack-work/install/x86_64-linux-tinfo6/lts-13.0/8.6.3/lib/x86_64-linux-ghc-8.6.3/example-0-AX5rovEqDIA8Qj0gJCELXw
Registering library for example-0..

$ echo ':quit' | stack exec ghci
GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/taylor/.ghci
>>> Leaving GHCi.

Then build the project with Cabal:

$ stack exec -- cabal v2-build
cabal: Use of GHC's environment variable GHC_PACKAGE_PATH is incompatible with
Cabal. Use the flag --package-db to specify a package database (it can be used
multiple times).

$ stack exec --no-ghc-package-path -- cabal v2-build
Resolving dependencies...
Build profile: -w ghc-8.6.3 -O1
In order, the following will be built (use -v for more details):
 - example-0 (lib) (first run)
Configuring library for example-0..
Preprocessing library for example-0..
Building library for example-0..

Now see that you can not longer get a REPL through Stack:

$ stack exec ghci
GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
Loaded package environment from /tmp/example/.ghc.environment.x86_64-linux-8.6.3

<interactive>:1:6: error:
    Not in scope: ‘System.IO.hSetBuffering’
    No module named ‘System.IO’ is imported.

<interactive>:1:30: error:
    Not in scope: ‘System.IO.stdin’
    No module named ‘System.IO’ is imported.

<interactive>:1:46: error:
    Not in scope: data constructor ‘System.IO.NoBuffering’
    No module named ‘System.IO’ is imported.

<interactive>:1:70: error:
    Not in scope: ‘System.IO.hSetBuffering’
    No module named ‘System.IO’ is imported.

<interactive>:1:94: error:
    Not in scope: ‘System.IO.stdout’
    No module named ‘System.IO’ is imported.

<interactive>:1:111: error:
    Not in scope: data constructor ‘System.IO.NoBuffering’
    No module named ‘System.IO’ is imported.

<interactive>:1:135: error:
    Not in scope: ‘System.IO.hSetBuffering’
    No module named ‘System.IO’ is imported.

<interactive>:1:159: error:
    Not in scope: ‘System.IO.stderr’
    No module named ‘System.IO’ is imported.

<interactive>:1:176: error:
    Not in scope: data constructor ‘System.IO.NoBuffering’
    No module named ‘System.IO’ is imported.

Finally, just to drive the point home, show that removing the GHC environment file allows you to get a REPL through Stack again:

$ rm .ghc.environment.x86_64-linux-8.6.3
$ echo ':quit' | stack exec ghci
GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/taylor/.ghci
>>> Leaving GHCi.

[snoyberg]

Thanks for the repro. Can you confirm that setting the GHC_ENVIRONMENT environment variable to - fixes this problem?

[tfausak]

Yup, that works.

$ echo ':quit' | env GHC_ENVIRONMENT=- stack exec ghci
GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/taylor/.ghci
>>> Leaving GHCi.

[snoyberg]

Cool, thanks. I played around with setting that environment variable before, but it seemed to break builds (ie calls into the Cabal library). We can probably treat this like the GHC_PACKAGE_PATH environment variable and simply not set it during Cabal calls. It seems like a strange design in GHC to respect an implicitly discovered file over an environment variable, but it's worth putting in a workaround for support with all current GHC versions.

[snoyberg]

OK, one more bit of fun. Does it also fail for you when using GHC 8.2.2 and setting that environment variable?

Afaict, there's no environment variable setting that will work across all versions of GHC out there today.

[snoyberg]

I've opened an issue about this against the Cabal repo: haskell/cabal#5988

[phadej]

GHC_ENVIRONMENT=- is supported only with ghc-8.4.4 and later.

---

For that empty package, stack ghci doesn't work either. stack exec ghci works, as it doesn't -hide-all-packages.

---

@tfausak said

After a short while I decided to also build it with Stack.

I tried to stack build and stack ghci: the existence of .ghc.environment files (valid or invalid) doesn't seem to have any affect.
The only thing which breaks is stack exec ghci.

I think setting GHC_ENVIRONMENT=- for stack exec would be good enough work-around, though it works only with ghc >=8.4.4.
Yet, cabal-3.0 won't generate environment files by default (PR is merged), so the corner case would be quite tiny.
There's a chance, that cabal-N.M would generate .ghc.environment files would be generated again

it's not like like we can't change the default back once we've worked out the few remaining kinks.

But there aren't any issues, neither any other clear descriptions what those are.
I asked Herbert to create those before any changes are made.