If I publish a project, files (mostly units) which are included in subdirectories of the original project are not included in the published project, although I added them to the project in the project inspector. In other words, only the files in the project's main dir. are also found in the folder of the published project.
what copy / paste does not solve and Publish Project should solve is updating the paths of files to the folder to where the publish is done.What paths do you mean? "Other unit files" and "Include files" should only contain relative paths which are part of the project, thus moving the project to another place always works.
what is Publish Project intended to do?Yes, that is the goal.
in other apps (for ex. Reaper FM) project publish takes all files in the project and places them in a folder. the user can take that folder to another computer and continue the work there given it has the same instruments installed would obtain same melody.
I am referring to the path of a file from another folder, ex: project in folder A using one file B1 from folder B and one file C1 from folder C.So it would copy files B1 and C1 into the project folder. I guess it is doable.
Now my expectation is that when I use publish the end result would be folder D that will have all project files from folder A and files B1 and C1.
This will allow if i move the folder to another machine where i have the package dependencies installed to obtain no errors when compiling.
I am not saying anything about dependencies files in folder D, only the files listed in the Files tree in Project Inspector.
(now if you have here dependencies it means that you updated them and needing, isn't it?)
looking forward to a RC version i could help with the testing as I do not know how to apply the patch and no experience with trunk.OK. Just wait 10 minutes. I will modify the necessary units in Laz. 1.8.4
based on this i can say that your assumption of subfolders is not consistent with the current IDE functionality.I did not assume anything. :) I just see how the "publish project" feature is implemented. Unfortunately it wasn't designed for this kind of functionality, for example it doesn't care about project files(the ones you see in the project inspector). Simply searches the main folder and export files according to the include/exclude filters. By the way consider the following hypothetical situation:
What versions of FPC / Lazarus are you running, btw?
I just see how the "publish project" feature is implemented. Unfortunately it wasn't designed for this kind of functionality, for example it doesn't care about project files(the ones you see in the project inspector. Simply searches the main folder and export files according to the include/exclude filters.)I highlightened the important part. This is basically meaning this feature is useless, unfortunately. I mean, which (serious) project doesn't have sub dirs. at some point (at least for resource files, e.g. images, or what not). :)
PS:
1. Download attached zip
2. Extract zip to $(LazarusDir/IDE) folder(make backup first)
3. Rebuild Lazarus(Tools-->Build Lazarus with profile...)
4. Export your project
@GetMem
say you have project file B1.pas, B2.pas in folder /home/user/B and C1.pas in /home/user/C and C2.pas in /home/user/C/extra.
publish target is /home/user/publish.
folders /home/user/B, /home/user/C and /home/user/C/extra should not be copied to /home/user/publish, only the files B1.pas, B2.pas, C1.pas, C2.pas should be copied to /home/user/publish.
now B1, B2, C1 and C2 are in the publish folder. to work you just need to update the file name value for these files in the project lpi file in the Units section for each corresponding entry.
the exported project is no longer equivalent to the original one and that is fine. but it compiles also on a different machine given i have the same packages installed.
Now every project file is copied into a single "published" folder. Paths are renamed in OI.ATM this function does not yet seem to work. I tested it with a large project containing subfolders forms, frames, units. Only the files in the main project folder are copied.
SoftWare
﹂ MyProject
-- ReadMe.txt
﹂ MainDir
-- Project1.lpr
-- Project1.lpi
﹂ AnotherDir
-- SomeSrc.pas
From Lazarus POV the project directory is "MainDir" because it contains the projects main source and info files. Logically it is directory "MyProject" because all sources are collected under it in subdirectories.SoftWare
﹂ MyProject
-- ReadMe.txt
﹂ MainDir
-- Project1.lpr
-- Project1.lpi
﹂ AnotherDir
-- SomeSrc.pas
﹂ MyOtherProjectDir
-- SomeOtherSrc.pas
MyOtherProjectDir is not exported.Did you test it? It should be exported if SomeOtherSrc.pas is part of the project
The question is where MyOtherProjectDir folder should go? Inside the publish folder, at the same level as Maindir.Why at the same level? Now the directory structure is retained.
Did you test it?Yes. If everything is inside a big directory the publish feature works correctly. The paths are retained, the project files are exported. I was considering more extreme cases like this:
I was considering more extreme cases like this:Their common parent is d:\ root. Thus the new logic should copy them to
Folder1: d:\ProjectFolder1\
Folder2: d:\ProjectFolder2\
Alternatively you can add subfolders too. If you wish to publish to d:\published\ for example, the feature will fail. Even in theory I don't know how the publish should work in this particular case, where the two folder should be copied?
Can you please debug the code to see what is going on. Add more DebugLn calls to procedure AdjustTopDir for example. I currently can develop for Windows only with a virtual machine and it is a big pain.In unit publishprojectdlg.pas, method AdjustTopDir instead of:
Both lib and backup are redundant for publishing purposes and should be ignored or I'm I missing something?You missed the ToDo comment in TPublishProjectOptions. :)
You missed the ToDo comment in TPublishProjectOptions. :)Indeed. :)
Could you please study and implement the filtering for those libs.Yes, but only next week. In my opinion we should ignore those folders completely. The publish feature is about creating a "clean" version of the project, only source code, no binaries, backup files, etc...
Hmmm... how did the old code handle those directories?The old code ignored every subdirectory, so those folders were never published.
BTW, why do you use the filters for your project? Are there some files which do not get copied otherwise?Until now I never used the publish project feature :), but I will in the future, especially when the compressing part is ready. In my opinion the Exclude filter is no longer needed, since we exclude everything, except project files. The include filter is useful if the user wish to publish other non-project files like images, resources, etc...
In my opinion we should ignore those folders completely. The publish feature is about creating a "clean" version of the project, only source code, no binaries, backup files, etc...Yes of course. I was only thinking how to best inspect if a certain file is under the output/backup directory. Now we extract "lib" or "backup", it is better to get the whole path and check if a file's path matches.
The old code ignored every subdirectory, so those folders were never published.Ah yes. It was very limited.
In my opinion the Exclude filter is no longer needed, since we exclude everything, except project files. The include filter is useful if the user wish to publish other non-project files like images, resources, etc...I guess in the old design either exclude or include filter was used at one time. It did not make much sense to use them both, unless the wildcard filters somehow overlapped. (?)
I implemented the remaining functionalities(r. 58851):Ah yes, you had the full commit rights already.
2. Update relative paths in the published lpiWhy is this needed? Is it only for the weird directory structures spreading over c:\ and d:\ drives? I don't think we should even support such weirdness.
Why is this needed? The file references are always relative AFAIK.Unfortunately no, not on windows at least. If you use the "Add files from the filesystem" and the file is not in the current directory, Project Inspector will add the full path of the file(see attached image). You don't have to take to the extreme, like spreading the files to c: and d: drives.
Otherwise paths in .lpi should not need adjustment because the directory structure is always kept.Since the lpi contains absolute paths, like the one in the screenshot, taking the published folder to another computer where the particular file does not exists, will lead to load/build failure. This is why I update the lpi file. Every path becomes relative to the published folder, more precisely relative to the lpi file inside the published folder. Under Linux/GTK2(just tested) apparently the paths are always relative, which is good.
I don't think we should even support such weirdness.In my opinion we should support it. As the current thread already showed, people do weird stuff, even if it's not recommended. Ideally no matter how spread the project is, the publish feature should bring it to one common folder, preserving the directory structure, but I don't want to insist on this subject, I can easily remove the support if necessary.
FDestProjFiles seems redundant. It is filled by iterating project files: "for i:=0 to CurProject.UnitCount-1 do"FDestProjFiles contains the destination files, the one in the published folder, CurProject.UnitCount refers to the actual units. Again this is only needed when absolute path are used.
Why not iterate them again when needed? It is used for updating paths in .lpi which also may be useless
Why is another FindAllFiles() needed for the zipper? We already know which files were copied. BTW, FindAllFiles() uses the same TFileSearcher class that we already use.OK, this one is indeed redundant. FDestProjFiles can be used to construct the zip file.
Why all the UpperCase() calls? The code in Lazarus dealing with units, projects and packages does not UpperCase everything yet it works. What happens if you remove them?Call me paranoid, but I always compare strings with upper/lower case. Most likely will work without uppercase too.
That is a bug. This is a cross-platform system, projects and packages should be portable between computers and OSs even without the publish feature.QuoteWhy is this needed? The file references are always relative AFAIK.Unfortunately no, not on windows at least. If you use the "Add files from the filesystem" and the file is not in the current directory, Project Inspector will add the full path of the file(see attached image).
OK, this one is indeed redundant. FDestProjFiles can be used to construct the zip file.+ the filtered files. FCopiedFiles has them all, if you adjust the path.
Call me paranoid, but I always compare strings with upper/lower case. Most likely will work without uppercase too.We should not have useless calls. It makes the code more difficult to follow and understand. I can test without UpperCases later.
In my opinion we are wasting to much time on a feature that apparently only we are interested. :)No, I believe it will be a very popular feature once it works well. For some reason nobody else has commented about it yet ...
3. Removed FDestProjFiles(I'm not sure about this, the code looks better, but since I have to do a lot of stringreplace to get the destination files, it's much slower)StringReplace is used in 2 places with identical code :
As a conclusion the Publish project feature only fails in extreme cases. Fixing the absolute path(windows only) issue, will improve the feature even more. I attach the files from r. 58851 as future reference, if somebody wish to handle extreme cases on windows.r.58851 can also be found in revision control easily. There is no reason to avoid revision control tools. They are such a good invention.
Now it creates a recursive loop. I will think how to fix it. Maybe it must be disabled after all.There is no recursive loop. Initially the project files are copied in the published folder, which is a subfolder in this particular case of the source directory. Then when IDE search for extra files will find the already copied project files(filter match *.pas files), then fails to copy them again since the source and the destination are the same. The FCopyFailedCount gets bigger then zero and the Run method exits without giving any clear feedback to the user. We need to add more message dialogs. Here is the antidote:
It must be allowed, yes. Wait...
... Now it creates a recursive loop. I will think how to fix it. Maybe it must be disabled after all.
Yes I believe the subdir problem can be solved. Now however I keep a small pause from this publish thingy and do other things. Maybe some of you will find a clean solution for it.OK. Fixed in r.58876. Publishing to a subfolder is still possible. Please test. Thank you.
OK. Fixed in r.58876. Publishing to a subfolder is still possible. Please test. Thank you.No it is not possible. The same recursive loop still happens. The "Published" directory is created under the project dir which then contains the project + another "Published" directory which then contains the project + another ... and so on until the OS is out of memory and kills Lazarus.
No it is not possible. The same recursive loop still happens. The "Published" directory is created under the project dir which then contains the project + another "Published" directory which then contains the project + another ... and so on until the OS is out of memory and kills Lazarus.Please try one more thing, add the following two line just below Result := mrOk(TPublisher.CopyAFile).
I attach the project I am testing with. It (obviously) has source files above the project's main dir.
It is (obviously) zipped using this new great publish feature, although not to a project's subdirectory. :):) It was harder to implement then I initially thought.
For me, R58877 publishing to a /published subdirectory works well, whether compressed or not.Thank you. Hopefully the publish project feature will become bug free soon.
Does just what it says on the tin.
Please try one more thing, add the following two line just below Result := mrOk(TPublisher.CopyAFile).Pos() works and AnsiStartsStr(), but the right place is method DoFileFound. You already had a test there but it was wrong.
Hopefully the publish project feature will become bug free soon.Now it is bug free, at least for projects! 8)
GetMem and howardpc, how come you were not able to reproduce the recursion problem?I did test your project, I saw no recursion issues. Most likely the Pos function was already in place, preventing infinite loops. The bottom line is: programming all days long does not help in spotting bugs. :)
Did you not test with projects having many directory levels, some of them above the project main dir? GetMem did such tests earlier I guess.
Can't you reproduce the problem even with my test project?
Pos() works and AnsiStartsStr(), but the right place is method DoFileFound. You already had a test there but it was wrong.OK. Thanks.
I fixed it in r58878. I also removed the now obsolete define.
Now it is bug free, at least for projects! 8)Well I'm not so sure about this. Let's wait for a few days...I would say is stable enough for testing. :)
Which Lazarus version?Version #: 2.0.0