Doom9's Forum - View Single Post

ultim · 1st November 2013, 22:28

Finally, some time to write down things on my mind. Well, not all the things, but it's a start

Everybody hates MT-modes, me included. They shouldn't be required, but unfortunately we cannot dispose of them, because there are a lot of old plugins out there that were written without any consideration for multithreading. Not supporting MT modes would mean throwing out a lot of great plugins, keeping only those that perfectly adhere to the points in my previous "Plugin Writing Tips" post. So I've been wondering, what MT-modes should be supported? We know Avisynth-MT supports at least 5, but which of those are really usefull? I have recently recognized that the Avisynth-MT OP post has been updated, and now it only lists modes 1-3 as "usefull". I've set out to compile my own list, while in the process answering questions such as "When exactly do we need those modes?", "What modes are needed if no plugins are fixed?", "What modes are needed if all plugins are fixed (as per my plugin writing tips)?" and also "Which plugins cannot be used with the recommended set of MT-modes (1-3) of Avisynth-MT"?

I started to think analytically, and compiled a set of three tables found here to summarize my findings. I might have overlooked some things or made a mistake or two, so let me know if you find anything. Each table lists four negative plugin properties as their columns, basically violations against multithreading support. The last column contains the protection that Avisynth+ would need to implement to keep that kind of plugin working correctly despite its violations.

Table 1. has all the different combinations of properties, irrespective of if that combination is even realistic at all or not. For example, there can be no plugin such as its 9. row, because a plugin can only possibly require ordered frame retrieval if it stores at least some kind of internal state. There are also cases of inherently broken plugins that cannot be guarded against in Avisynth, because they'd require limiting the plugin simultaneously to both single AND to multiple instances. So, let's see, how many MT-modes do we need? Six MT-modes, ouch. More precisely, six MT-modes if we do want to support all possible kinds of "broken" plugins, without fixing any of them. This is where I've said "no way I'm gonna implement all those MT-modes"! So what can we do? Fix the plugins of course!

One of the easiest plugin fixes is getting rid of storing the environment pointer in classes or globals. Fixing these violations has some nice advantages: First of all, this is a trivial fix in any plugin, so as long as we have the sources, this kind of fixing will go fast and easy. Also, quite importantly, getting rid of this single violation in every plugin will effectively ensure that there is at least a way to use all plugins in multithreaded modes (as long as the required protections are implemented in the core). So, never store the env.ptr. in classes and globals, and we are down to only 7 cases and 4 MT-modes (3 protections). Welcome to table 2. The protections in table 2. are what we'd need to have if we could perform only a trivial fix in the few plugins that need it.

Getting to table 3. from here is actually quite simple, only two simple facts have been used for this transformation. One is to get rid of using global variables. This is also a simple fix, as basically all that needs to be done, is to make class variables out of global variables. This will not effect plugin behavior at all, but it will remove the need to serialize these plugins, which is practically the same as letting them run fully parallel. Believe me, this fix is almost just copy-paste work for many plugins (not for those whose algorithm requires it, but those can be fixed too in every case, except that more work needs to be done). The main obstacle will (again) be having the sources to the offending plugins. OK, so far so good. But to get to table 3., I've also done something else: I've assumed that there are no plugins that must serially access all frames up to frame n, to calculate frame n. Why? Because such a plugin would hardly be usable in Avisynth (as it would make it impossible to seek inside or trim the video), and thus I actually doubt that one like that exists at all. (EDIT: I've been let to know that there are such plugins in existence. But all Avs+ could do to protect them is to play back all frames of the video internally until the requested frame is reached, which is performance-wise not practical at all - think of having to wait for multiple hours just to seek into the middle of the video clip).

So, we're at table 3., and IMHO have don't need any complex plugin fixes. Table 3. summarizes the MT modes that we need to implement if we assume we can fix all plugins until at least none of them uses global vars. Is that realistic? Yes, as long as we have the sources. Surprisingly to me, the conclusion in this table is that we either need no protection at all, or a single protection for unfixed plugins that store state across frames, but that still allows for full performance/parallelization, only with memory overhead (fully fixed plugins never require any protection from Avisynth). A second protection (3rd mode) is added to optimize source filters further, and while tbh I don't think it is strictly needed for correct source filters, it should still help their performance by reducing disk seeks. Another (important but sad) advantage of having this special mode for source filters, is that the same protection also lets us use unfixed plugins that use global or static variables. Note that while this special mode allows you to use such plugins, you should not rely on this mode (except for source filters), because it will cripple your multicore performance and flush it down the toilet, instead it is better to work on the plugin to not require a single serialized instance.

Avisynth+ will likely implement the modes listed in table 3., and I will take time to try and fix/modify as many plugins as possible, as well as motivate others to do so. In contrast to Avisynth-MT though, Avisynth+ will also allow plugins to register themself for the correct MT-mode they need, so users won't have to know by heart for all plugins if they have been fixed, or if they are unfixed, which mode they need.

Congrats for reading this rant of mine. What's to take home ?
- Fix plugins that store and reuse the env. ptr. It is often neglected but very important.
- Fix plugins that use global or static variables. We can guard against them to ensure they will work correctly, but they cannot be executed in parallel, and will seriously limit the multithreaded performance for your whole filterchain (even for correct plugins, due to frame dependencies).
- You can violate multiple rules for multithreading, and not all are equally important. Most importantly, ensure that the env.ptr. is never stored and reused. If you still have energy left after that, fix the plugin further so that no global/static state is used. Then if you still have energy left, try to turn all class members into read-only.
- The three MT-modes listed as "usefull" for Avisynth-MT really are the three most usefull MT-modes.
- The MT-modes of Avisynth-MT are not enough to use every single plugin when MT-mode is enabled. You might get crashes or corrupted frames if you use an unsupported plugin in an MT-mode, but don't blame Avisynth-MT, it is really not his fault. Blame the author of the offending plugin.
- Always fix plugins instead of relying on MT-protections. Most fixes are very easy and you will get much better performance than using a built-in protection.
- If you have the sources to a plugin, please don't keep it to yourself.

1st November 2013, 22:28	#204 \| Link
ultim AVS+ Dev Join Date: Aug 2013 Posts: 359	Finally, some time to write down things on my mind. Well, not all the things, but it's a start Everybody hates MT-modes, me included. They shouldn't be required, but unfortunately we cannot dispose of them, because there are a lot of old plugins out there that were written without any consideration for multithreading. Not supporting MT modes would mean throwing out a lot of great plugins, keeping only those that perfectly adhere to the points in my previous "Plugin Writing Tips" post. So I've been wondering, what MT-modes should be supported? We know Avisynth-MT supports at least 5, but which of those are really usefull? I have recently recognized that the Avisynth-MT OP post has been updated, and now it only lists modes 1-3 as "usefull". I've set out to compile my own list, while in the process answering questions such as "When exactly do we need those modes?", "What modes are needed if no plugins are fixed?", "What modes are needed if all plugins are fixed (as per my plugin writing tips)?" and also "Which plugins cannot be used with the recommended set of MT-modes (1-3) of Avisynth-MT"? I started to think analytically, and compiled a set of three tables found here to summarize my findings. I might have overlooked some things or made a mistake or two, so let me know if you find anything. Each table lists four negative plugin properties as their columns, basically violations against multithreading support. The last column contains the protection that Avisynth+ would need to implement to keep that kind of plugin working correctly despite its violations. Table 1. has all the different combinations of properties, irrespective of if that combination is even realistic at all or not. For example, there can be no plugin such as its 9. row, because a plugin can only possibly require ordered frame retrieval if it stores at least some kind of internal state. There are also cases of inherently broken plugins that cannot be guarded against in Avisynth, because they'd require limiting the plugin simultaneously to both single AND to multiple instances. So, let's see, how many MT-modes do we need? Six MT-modes, ouch. More precisely, six MT-modes if we do want to support all possible kinds of "broken" plugins, without fixing any of them. This is where I've said "no way I'm gonna implement all those MT-modes"! So what can we do? Fix the plugins of course! One of the easiest plugin fixes is getting rid of storing the environment pointer in classes or globals. Fixing these violations has some nice advantages: First of all, this is a trivial fix in any plugin, so as long as we have the sources, this kind of fixing will go fast and easy. Also, quite importantly, getting rid of this single violation in every plugin will effectively ensure that there is at least a way to use all plugins in multithreaded modes (as long as the required protections are implemented in the core). So, never store the env.ptr. in classes and globals, and we are down to only 7 cases and 4 MT-modes (3 protections). Welcome to table 2. The protections in table 2. are what we'd need to have if we could perform only a trivial fix in the few plugins that need it. Getting to table 3. from here is actually quite simple, only two simple facts have been used for this transformation. One is to get rid of using global variables. This is also a simple fix, as basically all that needs to be done, is to make class variables out of global variables. This will not effect plugin behavior at all, but it will remove the need to serialize these plugins, which is practically the same as letting them run fully parallel. Believe me, this fix is almost just copy-paste work for many plugins (not for those whose algorithm requires it, but those can be fixed too in every case, except that more work needs to be done). The main obstacle will (again) be having the sources to the offending plugins. OK, so far so good. But to get to table 3., I've also done something else: I've assumed that there are no plugins that must serially access all frames up to frame n, to calculate frame n. Why? Because such a plugin would hardly be usable in Avisynth (as it would make it impossible to seek inside or trim the video), and thus I actually doubt that one like that exists at all. (EDIT: I've been let to know that there are such plugins in existence. But all Avs+ could do to protect them is to play back all frames of the video internally until the requested frame is reached, which is performance-wise not practical at all - think of having to wait for multiple hours just to seek into the middle of the video clip). So, we're at table 3., and IMHO have don't need any complex plugin fixes. Table 3. summarizes the MT modes that we need to implement if we assume we can fix all plugins until at least none of them uses global vars. Is that realistic? Yes, as long as we have the sources. Surprisingly to me, the conclusion in this table is that we either need no protection at all, or a single protection for unfixed plugins that store state across frames, but that still allows for full performance/parallelization, only with memory overhead (fully fixed plugins never require any protection from Avisynth). A second protection (3rd mode) is added to optimize source filters further, and while tbh I don't think it is strictly needed for correct source filters, it should still help their performance by reducing disk seeks. Another (important but sad) advantage of having this special mode for source filters, is that the same protection also lets us use unfixed plugins that use global or static variables. Note that while this special mode allows you to use such plugins, you should not rely on this mode (except for source filters), because it will cripple your multicore performance and flush it down the toilet, instead it is better to work on the plugin to not require a single serialized instance. Avisynth+ will likely implement the modes listed in table 3., and I will take time to try and fix/modify as many plugins as possible, as well as motivate others to do so. In contrast to Avisynth-MT though, Avisynth+ will also allow plugins to register themself for the correct MT-mode they need, so users won't have to know by heart for all plugins if they have been fixed, or if they are unfixed, which mode they need. Congrats for reading this rant of mine. What's to take home ? - Fix plugins that store and reuse the env. ptr. It is often neglected but very important. - Fix plugins that use global or static variables. We can guard against them to ensure they will work correctly, but they cannot be executed in parallel, and will seriously limit the multithreaded performance for your whole filterchain (even for correct plugins, due to frame dependencies). - You can violate multiple rules for multithreading, and not all are equally important. Most importantly, ensure that the env.ptr. is never stored and reused. If you still have energy left after that, fix the plugin further so that no global/static state is used. Then if you still have energy left, try to turn all class members into read-only. - The three MT-modes listed as "usefull" for Avisynth-MT really are the three most usefull MT-modes. - The MT-modes of Avisynth-MT are not enough to use every single plugin when MT-mode is enabled. You might get crashes or corrupted frames if you use an unsupported plugin in an MT-mode, but don't blame Avisynth-MT, it is really not his fault. Blame the author of the offending plugin. - Always fix plugins instead of relying on MT-protections. Most fixes are very easy and you will get much better performance than using a built-in protection. - If you have the sources to a plugin, please don't keep it to yourself. Last edited by ultim; 1st November 2013 at 23:15.