SyMenu - SyMenu Suite - The SPS scripting engine - Messages

Message from Gianluca

Mon, 24 Jun 2019 18:30:28 GMT

chef wrote:

So do you recommend me starting to create such a SPS package for learning (and maintaining) a SPS package?

If you are interested in understanding the SPS technology and offering a contribution of this kind to this community, sure I do.

Message from chef

Mon, 24 Jun 2019 17:38:47 GMT

Gianluca wrote:

The dependencies release with a certain package is not our business. As SPS editors we release the packages as they are released from their authors.
Since the AutoIT major is still stuck on the 3.3.14.5 (https://www.autoitscript.com/site/autoit-script-editor/downloads/) I can't update it or update one of its dependency (SciTE).

This I already learned yesterday when following a recommended reading where you refer of this forum into a thread of another forum in the States.

And as far as I understood, the issue of package management including such dependencies one of the moderators claimed as still open meaning not addressed in a portable manner by any implementation or proposal in that other forum. This means that neither you nor any competitor had addressed this aspect yet. Nobody in that threat made a different claim.

Gianluca wrote:

If in your opinion a separate SPS package for SciTE is needed you can try to create it by your own and eventually share it with the community.

My post was in answer to the questions raised by sl23. He elaborated on handling of packages requiring another download. In that context, he raised several questions with seemingly not getting a reply yet. So I replied on his question if the editor mentioned by him works also without that context of the other package or only in that context. My reply was that it works also without that context.

I continued that the option to have a seperate SPS package for SciTE makes sense as it works also stand alone. Priority would probably not be high as there exist already editors with syntax high lighting in the suite although not yet SciTE as stand alone package.

Before I replied to sl23, I looked up in the suite and found that the situation is still as he reported and no seperate SPS package for SciTE created although another SPS package exists which extends the one included as reported by him for use in the context of another tool.

Gianluca wrote:

If in your opinion a separate SPS package for SciTE is needed you can try to create it by your own and eventually share it with the community.

I'm always open to new SPS editor and always available to help the newbie!

I already new this openness. (You mentioned it in another topic in the forum on a different request to another thread author.)

And as clarified in this response, I see no such need in the sense of requirement. I consider it just an option which makes sense.

So do you recommend me starting to create such a SPS package for learning (and maintaining) a SPS package?

(And I see and learned now that there exists a work around to the limitation of the forum software to support my style of quoting reported in another threat.)

Message from Gianluca

Mon, 24 Jun 2019 09:09:57 GMT

The dependencies release with a certain package is not our business. As SPS editors we release the packages as they are released from their authors.
Since the AutoIT major is still stuck on the 3.3.14.5 (https://www.autoitscript.com/site/autoit-script-editor/downloads/) I can't update it or update one of its dependency (SciTE).

If in your opinion a separate SPS package for SciTE is needed you can try to create it by your own and eventually share it with the community.

I'm always open to new SPS editor and always available to help the newbie!

Message from chef

Sun, 23 Jun 2019 19:41:12 GMT

sl23 wrote:

Can we specify an install path yet? If not then this needs adding. Also, double downloads.

For example, AutoIt3 has the SciTe, but, it seems, only the basic package. So, a second download would be required to install this package into:
AutoIT_sps\SciTe\ and overwrite the existing files there.

But is it necessary to have SciTe as a seperate sps in it's own right? Or, just include it with the AutoIt package? Is SciTe useful outside of AutoIt or only if used with AutoIt?

Just thinking, would it be easier for users if stage one was simply repeated? So simply keep the unpack box as is but have a second one for apps requiring double unpack? The second would need to be blank if not required.
edited by sl23 on 04/02/2016

Have you got any feed back on your question about SciTE?

I couldn't see one. I'm new to SyMenu. And I've used SciTE in the past on an older computer with Windows XP, not regularly. And I never used AutoIT. So it has been available seperately and was working seperately. There are various editors with syntax highlighting available. So adding a seperate package just for SciTE within SyMenu will increase the choice available to the users. So far, I've not seen a seperate SPS package for SciTE.

And another SyMenu tool tells me that the version of SciTE included with AutoIT on SyMenu is outdated with version 3.5.4. This other SyMenu tool tells me that other users of this tool have version 4.1.2 of SciTE installed. And another directory for software tells me that the author has already released version 4.1.7 of SciTE.

Message from sl23

Sat, 27 Feb 2016 19:31:28 GMT

Looking forward to this, thanks for your hard work, much appreciated ;-)

Message from Gianluca

Sat, 27 Feb 2016 11:22:14 GMT

Hello guys.

The new 5.01 version is in translation now so it will be released with a partial scripting engine: we'll only have the scripting before install. Sorry for that but the my release scheduling is too strict to implement the entire feature.

At least when SyMenu is released the before install scripting feature will be available an fully working (I hope).

I will update the SPS Builder just after the SyMenu release to avoid the publication of SPS that require the scripting ability before SyMenu is able to understand it.

Stay tuned.

Message from VVV_Easy_Symenu

Fri, 26 Feb 2016 22:05:27 GMT

Two other applications with "double zip"
10AppsManager -> http://www.thewindowsclub.com/downloads/10amW10.zip
Restart To UEFI -> http://66.226.78.22/downloadsite/Utility/Others/RestartToUEFI(v1.0.5).zip
Both are Windows 10 .. this is becoming a illness

Message from Gianluca

Mon, 08 Feb 2016 18:45:56 GMT

A deeper analysis is better than a superficial one. Usually when I have made complete draw of the problem I can take decisions and I can choose the more consistent solution.

This Paperscan is a pain... they offer a zip file to download (from here http://www.orpalis.com/download/paperscanfree.zip).
Inside the zip you'll find an executable. It's an InstallShield setup.
Inside the InstallShield setup you'll find an msi setup.
Inside the msi setup you'll finally find the software.
It's a weird system for distributing a software.

@VVV_Easy_Symenu
An external tool to unpack msi could be a good solution but it has to be silent, with a command line interface too, and it should be executed with normal privileges.
Is the tool you tested this one? http://www.jsware.net/jsware/msicode.php5#unpackx

Message from sl23

Sun, 07 Feb 2016 23:55:47 GMT

I have come across many examples of exe's wrapped in exe's, PaperScan Free is one such example. I used UniExtract on it but it doesn't work, at least the original and Gora's versions didn't, I'll test this new version 2 and see if that's different. It has had a major overhaul so maybe it will?!

But, I see what you mean Gian, I misunderstood the example! Analysing too deeply, I tend to do that!

Message from VVV_Easy_Symenu

Sun, 07 Feb 2016 21:08:04 GMT

Gianluca wrote: - to unpack an msi you need an admin command shell available.

I don't know how but Universal Extractor (SPS Suite) is capable to unpack ICE msi instalaton file without admin rights (7zip is not be able to unpack this rare file).
ICE-2.0.3-for-64-bit-Windows
I have done some inquiries and I understand in licenses "Universal_Extractor_sps\ docs\msi *. *" that it use a library of Total Commander or a msix tool (Universal_Extractor_sps\bin\MsiX.exe).
It's not in totaly silent mode but I hope this helps you.
edited by VVV_Easy_Symenu on 07/02/2016

Message from Gianluca

Sun, 07 Feb 2016 18:32:00 GMT

Maybe you are right but the request for a double or triple decompression comes from you editors
I'm only searching a good implementation that can grant flexibility, ease of use, and consistency. If some combinations won't be useful, who cares, the system theoretically covers any kind of combined compression for the supported formats.

Anyway I can give you other simple examples, besides yours, in which the double processing will be useful.

If you get a tar/gz package you can unpack it with SPS because the tar/gz package needs a double pass. I personally saw some apps packed with tar/gz.

Even more frequently you can find an exe installer (unpackable with 7z) packed inside an msi file, which is not directly supported but that sometimes is unpackable with 7z again. So you have another example where the double processing is useful.

The last one comes thanks to our friend VVV_Easy_Symenu.
Try to download this application.
http://www.tcxconverter.com/TCX_Converter/DOWNLOADS_files/TCX-C-Experience_2032_Windows.zip
It's a zip file that contains an innosetup package. Why the author decided to zip an already compressed setup package? Again who cares, maybe he didn't want his users have to download an exe file... we all know how scaring an exe file can be
The only fact here is that an innosetup package inside a zip package exists.

Message from sl23

Sun, 07 Feb 2016 13:06:25 GMT

7z | rar | zip | innosetup [relativePathToTarget]

Knowing nothing about scripting, this looks to me like it's saying:
1. Unpack 7z, then
2. Unpack rar, then
3. Unpack zip, then
4. Unpack innosetup to relative path.

Obviously this can't be the case, nobody packs things this way, though I've come across zip's in rar files, but never an app.

So what exactly is going on here?
edited by sl23 on 07/02/2016

Message from Gianluca

Sun, 07 Feb 2016 10:37:53 GMT

Hello guys.

I'm working hard on this new feature and my ideas is becoming clearer.

The first release of the scripting engine will be very simple with only basic features. We have to test it and to check if it is really useful before starting to fill it with several new features.

I'm currently working on the easier part, the script before installing.

The syntax will be easier than that one I have theorized before. Sure, with this approach you'll have less freedom in your scripts, but I absolutely have to grant security to the users. The priority is security, if I can't grant it I'll immediately abort the scripting engine project.

The before install condition is this: we have a just downloaded package that is located in a temporary folder on your PC. SyMenu knows exactly where this package is located and what its name is so you have to specify nothing because it is obvious that you are processing the package.
But you have to instruct SyMenu on what file you expected from the first decompression, because a package could contain more that one file with names different from the container.

Therefore the syntax becomes:
7z | rar | zip | innosetup [relativePathToTarget]

Examples:
7z insidePackageFile.zip
zip packageFolder\insidePackageFile.exe
innoSetup "package folder\insidePackage.zip"

I did some examples with inconsistent extensions among the unpacker and the file (7z/zip, zip/exe, innoSetup/zip) to explain better that the first unpacker is not applying to the argument but the argument is the result of the unpacker action.

A clearer syntax should be:
7z THEN insidePackageFile.zip
zip THEN packageFolder\insidePackageFile.exe
innoSetup THEN "package folder\insidePackage.zip"

I think that a shorter syntax is easier to understand and remember but let me know what do you think. I even could implement both of them.

The first release won't have the double unpackers feature but surely I'll implement it in a future version.
The double download is more and more difficult so it could arrive later.

Another news is about the msi format. I'm really sorry but I can't add that to the SPS, neither normal nor scripted, for two main reasons:
- to unpack an msi you need an admin command shell available. It's an incredible thing but it's true. I can't force a user to gain an administrative privilege to simply unpack a file;
- if you unpack an msi in silent mode and it hangs for whatever reason, the process remains active indefinitely. Try to unpack an msi in silent mode without admin privileges if you don't believe....

The more I go deep in the MS products the more I discover fantastic things and creepy piece of architectures. Sometimes it seems that MS is two different companies.....

Ok. I'll let you posted of next progress.

Message from VVV_Easy_Symenu

Sat, 06 Feb 2016 09:53:45 GMT

Some thoughts,
The future "introduce multiline string" SPS scriping feature, it will be before or after the "Update no Copy Files" actual SPS feature? Or this actual feature evolve a some script command?
I think that multiple dowloading will be a good feature, for instance for the separate translations files, plugins, etc. If you do this, why not evolve the actual feature "download URL" to a script command parameter? So,

Normal SPS: 7z {UrlApp} {AppSPSfolder}
Doble unpack: 7z{UrlApp} {AppSPSfolder}; 7z {AppSPSfolder\firspack} {AppSPSfolder}; del {AppSPSfolder\firspack}
Several dowloads: 7z{UrlApp} {AppSPSfolder}; 7z{UrlTranslation} {AppSPSfolder\translations}

edited by VVV_Easy_Symenu on 06/02/2016

Message from sl23

Thu, 04 Feb 2016 17:44:30 GMT

Can we specify an install path yet? If not then this needs adding. Also, double downloads.

For example, AutoIt3 has the SciTe, but, it seems, only the basic package. So, a second download would be required to install this package into:
AutoIT_sps\SciTe\ and overwrite the existing files there.

But is it necessary to have SciTe as a seperate sps in it's own right? Or, just include it with the AutoIt package? Is SciTe useful outside of AutoIt or only if used with AutoIt?

Just thinking, would it be easier for users if stage one was simply repeated? So simply keep the unpack box as is but have a second one for apps requiring double unpack? The second would need to be blank if not required.
edited by sl23 on 04/02/2016

Message from sl23

Wed, 03 Feb 2016 12:42:47 GMT

Oh yes, sorry, I remember now, doh!

Message from Gianluca

Wed, 03 Feb 2016 11:24:21 GMT

We were talking about a different thing, the future of the entire SPS architecture. I don't know if I can evern realize that project due to the long time and work required.

The scripting engine is a simply extension of the current SPS schema. Not a big thing.

Message from sl23

Wed, 03 Feb 2016 10:30:34 GMT

Gianluca wrote:

sl23 wrote:
BTW, is it just me or is there a problem with this site? text is running off the side of the page and I'm having to scroll right to read longer posts. I never had this issue before. It also happens on my android phone.

I think it's because your previous post where you put a really long line starting with "recent_docs =...". Try to modify it and shorten this line.

Ok, I deleted the entire code. See how it goes. Thanks for the tip!

Gianluca wrote:

sl23 wrote:
Any chance phase one, double unpack, will arrive in the first half of this year?

What? Absolutely not. I hope you can have the entire script engine within the end of March both in SyMenu and SPS Manager.

Seriously? I thought you said in email it would take 6-8 months of solid work, but would take longer due to other responsibilities? Did I misunderstand?
edited by sl23 on 03/02/2016

Message from Gianluca

Wed, 03 Feb 2016 07:51:21 GMT

VVV_Easy_Symenu wrote:

BTW attention to the special characters and syntax with the script interpreter and the SPS Container interpreter

Yes you are right I will pay attention with that encoding the entire script before writing into the SPS file. BTW the forum software is not mine

sl23 wrote:

BTW, is it just me or is there a problem with this site? text is running off the side of the page and I'm having to scroll right to read longer posts. I never had this issue before. It also happens on my android phone.

I think it's because your previous post where you put a really long line starting with "recent_docs =...". Try to modify it and shorten this line.

sl23 wrote:

Any chance phase one, double unpack, will arrive in the first half of this year?

What? Absolutely not. I hope you can have the entire script engine within the end of March both in SyMenu and SPS Manager.

Message from sl23

Wed, 03 Feb 2016 00:50:26 GMT

VVV_Easy_Symenu wrote:

With the first slot I can made SPS of:
"TCX Converter - The multiconverter tool", "Eye4" (packed twice)
"MS ICE" and enhance "FileBot" (msi instaler)

With the second slot I can made SPS of:
"UltraVNC viewer" (rename and delete files)
Erase the "$PLUGINSDIR" (directory of remaining setup files, OpenCandy, etc).
Perhaps make "UltraStar" automatically portable as you explain in the "Portable Emule"

I have around 25 more apps to add via sps once the script engine is in place! Can't wait to finally get all my apps auto updated, even if via my own sps, far easier than d/l, extract copy/paste, test!

Any chance phase one, double unpack, will arrive in the first half of this year?

BTW, is it just me or is there a problem with this site? text is running off the side of the page and I'm having to scroll right to read longer posts. I never had this issue before. It also happens on my android phone.
edited by sl23 on 03/02/2016

Message from VVV_Easy_Symenu

Tue, 02 Feb 2016 21:24:32 GMT

The SPS scripting engine is a very good new.

With the first slot I can made SPS of:
"TCX Converter - The multiconverter tool", "Eye4" (packed twice)
"MS ICE" and enhance "FileBot" (msi instaler)

With the second slot I can made SPS of:
"UltraVNC viewer" (rename and delete files)
Erase the "$PLUGINSDIR" (directory of remaining setup files, OpenCandy, etc).
Perhaps make "UltraStar" automatically portable as you explain in the "Portable Emule"

BTW attention to the special characters and syntax with the script interpreter and the SPS Container interpreter: For instance, try to add a message with the word Stealth inside <> in this forum .

Message from Gianluca

Tue, 02 Feb 2016 12:07:34 GMT

Yes I can but in my opinion this syntax is not elegant.
It's a matter of not reinventing the whell so maybe another syntax, clearer and smarter exists but since I don't know it, I'm asking for something better.

Message from sl23

Tue, 02 Feb 2016 11:11:53 GMT

SPSFileContentStart
SPSFileContentStop

Is there any reason this couldn't be used as is? Could you code it to look for these two strings to identify blocks of text?

Message from Gianluca

Tue, 02 Feb 2016 08:12:42 GMT

Sure it'll be part of the SPS Builder too.
In this way you'll be able to test an SPS even in the scripting part too.

I agree with you with a more simple system to introduce the file content and a change in command name too. Moreover an optional switch to define the encoding.
Since you'll be able to introduce multiline string we need a way to define a string terminator.

An hypothesis could be to use magic strings.

create an empty file in ANSI
createFile ANSI {app}\config.ini

create a file with one single line in UTF-8
createFile UTF-8 {app}\config.ini
SPSFileContentStart
unique line
SPSFileContentStop

create a file with multiple lines in default encodind (ANSI)
createFile {app}\config.ini
SPSFileContentStart
first line
second line
SPSFileContentStop

I don't like at all using magic strings as terminator but it is a good way to avoid escaping inside the text. If I use double quotes as terminator and you need to put double quotes inside the text the result will be something like this:

"first line
second line
var a = \"test\"
third line"

or worst

"first line
second line
var a = ""test""
third line"

Some other suggestions?

Message from sl23

Mon, 01 Feb 2016 19:46:22 GMT

Looks good, from the little I know about this stuff, but seems to perform all the functions we had problems with.
Will this be built into, and form part of the SPS Builder app?

Is there an easier way than this though:
create text file with values inside
echo.first value=1 >{app}\config.ini
append values to a text file
echo.second value=2 >>{app}\config.ini

When it comes to adding many lines to a config file, perhaps eventually the X-Launcher or I think it was either Cherrytree or Calibre that required a fully populated ini file. This could tedious in such cases. For example adding this to a blank Cherrytree config:
edited by sl23 on 03/02/2016

Message from Gianluca

Mon, 01 Feb 2016 18:25:48 GMT

Two words about the future scripting engine.

It should work in two different moments:
1) the first slot represents the moment just after the package download and before the "installation" (we know that SPS for real installs nothing...)
2) the second slot represents the moment after the "installation" and before the linkage to SyMenu.

During the first slot I'm thinking to only add commands to unpack the downloaded package. So you can write something like this:

7z {package}
rar {package}
zip {package}
innoSetup {package}
msi {package} (I know it doesn't exist anymore but it will do)

You can apply the unpacker more than once but you have no other command available.

During the second slot you have the app already copied in the target folder so here you need something more complex.
I'm thinking to limit the commands effect to the app root folder and any subfolders for security reasons. So you'll have a placeholder like {app} to refer to the root folder that must always appear.
You are not allowed to go under the {app} folder.
Inside this folder you have a large availability of commands.
I'm thinking to:

create dir
md {app}\bin\newFolder
rename dir
ren {app}\ben\newFolder {app}\ben\oldFolder
remove empty dir
rd {app}\bin\oldFolder
move/rename file
move {app}\whatever.exe {app}\whateverRenamed.exe
move {app}\whatever.exe {app}\bin\whateverRenamed.exe
copy/rename file
copy {app}\whatever.exe {app}\folder\whateverRenamed.exe
delete file
del {app}\whatever.exe
create empty text file
echo. > {app}\config.ini
create text file with values inside
echo.first value=1 >{app}\config.ini
append values to a text file
echo.second value=2 >>{app}\config.ini

As you've surely observed this is a Windows command subsets. So it is easy to imagine that I'll use the original Windows command to execute the custom instruction after the path conversion. Anyway even if the commands is executed by the Windows shell I'm thinking to forbid the command switches, always for security reasons. So you can only write the pure commands without parameters.

It's only a draft but I'm really interested in you points of view.

Let me know.