I spent the best part of half a day last week trying to get to the bottom of a very strange issue on a project I was working on. I had just created a new unit in the project and had reached a point where I needed to add that unit to the uses list of an existing unit in the same project. It was at that point that things started to go awry.
When it encountered the new unit in the uses list of the referencing unit, it complained that the file did not exist.
This was strange because I was sure that the unit was in the DPR uses list and so if there was any problem with it not existing this should have been encountered before. So I double checked, and sure enough the unit was in the DPR uses list.
The first thing I always check when I get spurious errors involving unit references is my project search path and the possibility that I might have some old DCU’s lurking in places that they should not be.
So I checked this and – as I thought, this being a new unit – this was not the problem. There were no DCU’s or old copies of PAS files where there shouldn’t be (no duplicates etc) and they were all up to date (full build’s were indeed building fresh new DCU’s). But, to be doubly sure I deleted all history files and all DCU’s anyway.
Nothing changed.
But, at least duplicate/old DCUs were eliminated as a possibility.
Next, I removed the unit from the unit uses list, and to make doubly sure that it was still being compiled by the project (by virtue of it being referenced in the DPR uses list) I added some code to the initialization section of the new unit, to throw up a message box at runtime, just so I could be certain that it was being compiled into the project.
Sure enough, the project compiled successfully and when it was run the message was presented as expected. Clearly this unit that the compiler was complaining did not exist not only did exist but was compiling without a hitch.
So, thinking that whatever SNAFU had occurred had perhaps now resolved itself somehow, I added the unit back to the other unit uses list:
“ERROR: File does not exist”
Aaagh! Ok, so what next ?
Well, the unit name was a dotted style name, a convention that I have embraced since Delphi 7 and though it has caused the occasional problem it has generally been a sound strategy. But in this case, there was a part of the unit name that might – at the edge of reason – conceivably cause some problem. The unit name was “<...>.Directory.Interfaces”.
The <...> part was common to a large number of other units in the same project and since this had not been a problem up to now I had no reason to think they might be involved in this case. Similarly “Interfaces” was part of a number of other unit names, with no problems to date.
“Directory”, on the other hand, was a new addition to the lexicon of unit name components. Even so, I had established that this was not causing problems in the DPR uses list so I could not envisage why it would be an issue in a unit uses list.
But, the first rule of problem solving: Eliminate the improbable.
So I went back to the beginning. I removed all references to my new unit and created another new unit with a simple, plain name: “Foo”. A true classic. π
Everything went as before. No problem when compiling this with a reference in the DPR uses list but as soon as I added it to the other unit involved:
“ERROR: File does not exist”
This was getting ridiculous.
OK, so remove the reference from this unit and add it to some other arbitrary unit instead:
“ERROR: File does not exist”
At this point I decided that my entire Delphi installation might have some deeper issues and the prospect of having to re-create my entire development environment began to lurk ominously in the shadows.
One last thing occurred to me to try: the DPROJ file.
I knew that it contained information about the project contents because this was the place where – among other things – the order of units in the Project Manager/Explorer was maintained.
But I also knew that aside from compiler and other project settings, the DPR was still the definitive reference for the project contents – or should be. Fortunately, the project had no particular specific options set. In particular, the search path referenced only a couple of 3rd party source trees (Indy, FastMM) which could easily be restored.
So I simply deleted the DPROJ. With interesting results.
Suddenly my project would not compile at all since it transpired that I had several units missing from my DPR uses list. These were not units that were previously on the project path because these were units that were part of the project’s own source tree which was not previously on the project path at all.
Equally I had already deleted all the project DCU’s so it couldn’t have been that the compiler had previously been picking up old DCU’s before now. The only thing that had changed was that the DPROJ had been deleted and now recreated from scratch (by the IDE).
Once I dealt with the missing units and had a compiling project once more, I added my new unit back in to the DPR and unit uses lists as required and hey presto. Everything was fine once more.
The lesson I learned from all this:
When experiencing perplexing and apparently impossible errors involving unit referencing in a Delphi project, investigate the possibility of a corrupt or damaged DPROJ before wasting time on other causes that will consume more time to explore.
I only wish I had kept a copy of the DPROJ so that I might have conducted a post-mortem on the damaged/corrupted file which might have shed some light on what had happened.
We had such problems over and over again since Delphi 2007 (the first version to use MSBuild). Each time we moved to a newer Delphi release it sometimes failed to upgrade its own project files and we faced the strangest problems as a result. Any attempt to debug the problem was swarted by the IDE by rewriting and rearranging the complete *.dproj file each and every time. It has become worse in XE3 which seems to change the file on each compile. As we store some lengthy search path in our project files and also some compiler defines I had to write a tool to change those files when needed. Anyway the only cure I have found to all the strange problems is to let the IDE recreate the file from scratch.
Did you compare the old and the newly created .dproj file? Maybe that could shed further light on the issue. I have had my own share of broken .dproj files and am no really surprised by your findings. Usually it was duplicate entries for a file that was renamed.
No, unfortunately I did not. π
-Searchpaths in the dproj may go haywire (it can suddenly find sources which should absolutely not be available and are not visible in the project options dialog, a nightmare).
-It can have mismatches between dpr and dproj. Seems to be some strange synchronisation problem between dpr and dproj)
-If opened by a newer delphi version the newer version adds new platforms to the xml older versions cannot handle (and the lower version is unable to compile anything, in this case XE).
So yeah, the ‘new’ dproj is a real advantage and is alway’s the first suspect when things go wrong (and has cost me an idiotic amount of time also)
DPROJ is clear violation of DRY principle…
When things get weird i launch SysInternals Project Monitor and look which files Delphi acutalyl searched for or read during compilation. Sometimes ti helps. For example when you spot that Delphi compiled the program against one BPL, and the program EXE is run against another BPL.
However, seeing error “file not found” i think the 1st question that programmer would as would be “what file?” and it is a bit strange that – following the text – Jolion was not curious.
Um, I’m not quite sure what your point is. I didn’t need to be curious. The full error message, as I would have thought was obvious, identified the file. Specifically the “<…>.Directory.Interfaces” unit (and then subsequently the “Foo” unit). I didn’t think that needed spelling out by quoting the error message entirely verbatim. Perhaps I was mistaken. π
i.e. the problem seems to be that the corrupt DPROJ guides the compiler in locating files referenced in units. The reference in the DPR respects any path included in an “in” clause (as was the case here).
Well, that part is a bit speculative since, as I say, unfortunately I didn’t think to keep the old, borked DPROJ around for analysis. My priority was getting my project compiled, not doing Embarcadero’s QA for them.
I’ve had very similar problems that prevented remote debugging of a large project. Deleting the DPROJ and letting Delphi recreate it eventually solved the problem but it was time consuming. IME comparing DPROJ’s fails because Delphi will often rearrange the XML nodes making comparison difficult. I wrote the http://delphi-divining.blogspot.co.uk/2012/10/dprojmaker-tool-to-create-delphi.html tool to help others with this problem.
Your tool is intriguing, but it’s not clear to me how to create an initial configuration. Must I do that manually, line by line?
It has been my unhappy experience that the dproj file can be the source of very quirky misbehaviors, as you discovered. My sense of it is that although there are things in the dproj which clearly are not in the dpr, the two should be in sync, yet all too easily can get out of sync.
Why it happens is a mystery, but it’s just one of those things which annoys, but has not yet annoyed me sufficiently to give me the inspiration Brian clearly had. I must try out DprojMaker.