Working on v0.8e

[KAMIKADzE]

Decided to start this thread, just to keep you somewhat updated on how things are going, as the last Launcher update was released quite some time ago...
In order to not make this post very long - a lot of things happened, and that somewhat derailed my Launcher development. At the moment I'm not working on any of the v0.9 functionality, at least not directly, but I do plan to get back to it eventually, though the focus is currently on other things. That said - there are a lot of changes and improvements, already implemented in Launcher, they just require a bit of polishing here and there before the actual release. Here's what is definitely planned for v0.8e as of now, unless there will be some kind of emergency need to release it faster:

• BDO PERS functionality, as promised there.
• New BDO Repack Mimic mods (2 mods, one of those I use for my patch for more than a month already, more on that - in the next status update post). Minor improvements to existing BDO Patcher's functionality.
• A lot of Console improvements and changes. New commands.
• Pre-configured, fixed, Power Shell with Launcher's Console integration. Optional installation.
• Some improvements to the installation itself.
• Minor changes to Log. Don't want to specify exactly, as I'm not sure what will be released, but essentially Log will receive a lot of new functionality in the future. I'm also considering whether to merge it with Net Monitor, but that definitely won't happen in v0.8e.
• Improvements and new algorithm in File Cryptor.
• The first release of Crypto tool (also promised somewhere). I've prepared its transfer to Launcher (from another program), but the amount of functionality opened within the first release is still under the question (still requires a lot of user-error management and the relevant UI).
• Windows KB check.
• Various improvements and things that I definitely forgot to mention.

Possible (most likely not v0.8e):

• Context Menu editor. Still not sure about the name of the tool, maybe something like Shell Editor, as it will eventually have functionality beyond Context Menu, maybe even within its first release.
• FLT and NT types to Log. Both will be available as Console commands, but I'm also considering adding them to GUI of renewed Log.
• Font selection for Launcher (default), Console and possibly renewed way of selection for BDO Patcher's Fonts tab. Along with it there will most likely be some default windows sizes changes.
• Text Editor. Improved Console will be basically the preview of it. It will be the recommended editor for things like FEX, Schemes, etc.
• Fix for the settings corruption, that can occur at the times like BSOD, unexpected crash or electricity loss.

There are more things in progress, but those mentioned above^ are relatively close to completion.
With how things are going right now - I plan to release it around the end of June, but my life is quite unpredictable, so who knows. It also significantly depends on the bombardment of events and changes in BDO, as both Launcher development and BDO share the available time and there's also that.
All in all - the upcoming update would be worthy of the title v0.9, if v0.9 was not announced already
Oh, and also I'm working on an auto-execution tool, that will do things auto-magically (what you tell it to do), and will be used by other tools to automate things. For example, I've moved to Office365, for unrelated reasons, and there are 2 garbage services, 1 of which closes itself, after some time, while the other never does (when all of the Office programs are closed), so at the moment I have a keys combo bound in Console to close them, which is fine, but still requires an extra step, while this tool will be basically a set-and-forget option + free an extra keys combination. That was just one example, I even consider moving BDO Auto-Patcher to it, although I'm not sure yet (UI to toggle it will remain in BDO Patcher). The main reason for my consideration of consolidating automation tasks is to reduce CPU usage, but in either case the tool itself will be useful for other purposes, especially since it'll come with a new Watchdog available, that instead of timed pulling will subscribe to system events, like Process start, which is significantly slower to trigger, but is essentially negligible in terms of CPU usage, and, while it wouldn't be ideal for BDO Auto-Patcher, it is a perfect option for things like stopping pesky services or processes.
And that's still not everything that's planned and in the making at such moments, when I think about updating user guide in help menu...
And I know what many of you will think - "just focus on one thing at a time and release it", but the problem is that I'm mostly working on Launcher in waves (periods), rather than continuously, and after a while I need to review what was done in order to continue the work (that also takes time), or I can leave it half-working and continue to use it in that state, while not wasting what little time I have and working on something else, and so I end up with many things "in-progress" . Technically I could release them in such state (actually, maybe like half of them), but I would waste my time on that + many things don't even have GUI for average user, not even mentioning the user-error prevention = forum filled with useless error and bugs reports, and a terrible experiences for users. Thanks for understanding why I prefer to not release unfinished functionality.
P.S. From time to time I also run certain benchmarks on code and optimize it for better performance, which you see as "improvements", but at the moment I'm considering of degrading performance, possibly for the 1st time, for a lot of common BDO Patcher's functionality. Theoretically it should be a very tiny unnoticeable regression, maybe I'll even break it even in the progress of doing that. Why? Most of the BDO Patcher's functionality for different servers is essentially a set of multiples. Initially there were many differences between the servers and that was a wise choice to keep them separated from each other, in order to avoid any emergency rewrites, but as the time progressed - new servers were added, they were becoming more and more alike, I've added more and more functions, and at the moment there aren't that many differences. I want to combine it all into a single set of functions, that will be configured by server-specific settings, which isn't that much different from what is going on now, basically I'll trade the amount of functions with amount of settings passed around, which will result in a more manageable codebase (no need to write the same, slightly different thing for all the servers out there) + it will allow me to add the option of adding extra servers via settings (so that you will be able to add new server, in case there will be one, and I won't be interested in BDO at that time, or something like that). The thing is, if I would not write about it - you would never notice the difference, although there definitely will be, at least while I'm in a process of consolidating them all. I'm still in consideration of the least impactful way of doing all of that, as I definitely won't manage to do it all in one go, but the main point of writing all of this is that if you'll stumble upon a bug in the old functionality for your server - let me know (in reality it should be the other way around - less possible bugs, since you will run through the same code as the primarily tested one - JP2). I'll try to start it all with Repacking, since currently Mimic mods are available only for JP2, instead of adding it to all servers I'll do it in reverse (not promising, depends on how it'll go).

[KAMIKADzE]

Now onto the main reason for why I started this thread - new BDO Repack Mimic mods.
There are actually 3 mods, one which has 2 sub-mods, but for the sake of configuration simplicity (it's already quite complex) they all fall under the same category of Mimic mods (4 in total). Those 2 sub-mods are the actual useful ones, hence the reason why I've mentioned only 2, I'll briefly mention the other ones, but they should not be used and might not be released.

Before we'll move on to the explanation of mods, let me explain to you why and who should use them, starting with why? About a month ago (give or take), on top of the existing verifications, PA added file size verification (always wondered, why there was none, and they used more complex checks first, expected to see it way way sooner), which wasn't a big surprise to me, hence the reason why my patch users haven't even noticed that (afaik some servers haven't implemented this check yet, so if the old mimic works for you then it works, both of the new N ones will work as well).
The main difference with the modless Mimic is that the Mimic without mods appends extra data to the file, keeping the original intact (excluding user modifications), while with mods it writes extra data inside the file, keeping the original file size. Now it should probably be obvious to you, that writing data inside the existing data of the file means that you will rewrite something, and, unlike your modifications to files xyz, this written extra data is exactly that - extra, meaning that you have to remove some of the existing useful data to clear up some space in this size-restricted file. Hopefully that makes it clear, why there was no such problems before. All of these mods are essentially a different approaches at clearing space for extra data, and there are even more options available (few more mods can be added, if need be). Let's take a look at them, shall we?:

• OPFN. I don't even remember what I was testing with it. After reviewing the code I'll probably remove that one.
• CL. That is your nuclear tank approach, it just rides in and writes the data. Totally BAD approach, but requires no additional settings, and might work if many things align properly for it to work. It is a great example, of why we will use one of the 2 following N mods instead.
• NDFN and NFFN. Both of them are basically the same N mod, with slightly different way of configuration. In order to reduce the wall of text - I'll explain them together, primarily focusing on NFFN, which I prefer to use.
  Essentially with N approach we reduce the amount of actual data by shortening one or more of the available file names (FN in the name of mod), leaving space for BDO Patcher to work with (it will work regardless, but more on that later). Let's view this on the example of NFFN in FEX:
  

  Code: Select all

          "MetaChecksum": "Mimic",
        "MetaChecksumMod": {
          "ModType": "NFFN",
          "Names": {
            "guidemap_east.dds": "e.dds",
            "guidemap_west.dds": "w.dds"
          }
        }
  
  Here^ you can see how I rewrite 2 file names from their original names to the reduced ones.
  Now what are those files and why them? You can read my short post about them THERE, in short - they are the files that are never used by the client. Rewriting the names means essentially breaking the link between files and the game client, so you want to do it either with the files that are never used, or that are virtually never used (like for some events of the other server). How do you find such files? - use Get files list in BDO Patcher's FEX tab, I've explained multiple times on how to use it, so if you're new either use search on the forum or check guides (like THAT one or THAT one).
  What will happen if you rewrite the name of the file that is actually used by the game? - nothing critical, either the game won't start (complaining about file corruption or something like that) or it will self-close (with similar complain) later on, during the play, so you'll know if you've made the wrong choice.
  As to how you should rewrite the name - it's up to you, just try to avoid naming the file to something that already exists in the files list, as another example - you can rename them to unused extension, like "1.x", "2.x", etc. or you can strip the extension, if you're still unsure - just search over the files list, if you can't find a match - you're good to go.
  Do note (check the example above^), that you rename only the file names (text past the last slash /), as unlike of what you can see in the files list, in meta itself - folder and file names are stored separately.
  Now to the most important part - BDO Patcher requires a certain amount of bytes to write that extra data. I will not bother you with the mathematical details, but essentially it's somewhere between 12 and 23 bytes, depending on the original meta. I'll add CCExtra menu option in ??? menu, that will have some extra data on top of the current CC option + the required bytes number for tested meta, but that's just an option for ocd people, for everyone else I recommend going with my approach, shown in the example, and rewriting 24 characters of text (it can be 1 file, 10 files, or even 20, doesn't matter), which will result in a reduction of 24+ bytes (depends on multiple things; this approach is basically a set and forget). In case you'll go the try harding route, optimizing each time - I recommend to stick to the 4 bytes rule (rewrite 4x chars), as in the way encryption and overall writes work (basically a lot of math) it'll shift depending on multiple conditions and you might not try hard enough and create more space than required xD it's also the reason why meta size is always dividable by 4 (at least up until now, but PA can always add an extra byte of some data or something like that, in the future).
  Even in case when you leave not enough of space for BDO Patcher - it will repack regardless, and that is an intentional feature (I do not throw big red windows at your face, for multitude of reasons). Such repack will overwrite the last file name, making your game either complaining at start or after load, depending on the rewritten data (last file is usually wwise_ids.h, which is a list of audio-related things, so the game will complain about it rather fast, it's also the reason why CL mod is BAD; note that I have written not about the data that is being rewritten, but the data BDO Patcher rewrites, as there is a finishing touch for the file name data to properly load, and most likely it won't be the same as the starting data of what is written for mimic).
  Almost forgot to mention the difference of NDFN mod - instead of manually renaming the file name, and counting the characters, it allows you to rename by removing the amount of first characters (if we were to apply this mod to our example in the same manner, by specifying value of 12 and removing 12 characters from each - it would result in duplicate entry of "t.dds", but nothing stops you from removing more characters from one of them and less from the other, I just personally prefer the option with more precise control).

I think I've mentioned everything I wanted to. In reality there is, of course, way more math involved in mods, than in modless version, but for you it will be practically indistinguishable (other than the resulting files size). I've also added automatic time copying, just in case they decide to check times as well (you probably noticed that, if you use my patch). The speed of existing repacks will also increase, as I've made some improvements. That's about it, for now you have plenty of time to check files list and investigate on the possible file names that you can safely rewrite.