Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
![]() |
#1 | Link |
Registered User
Join Date: Jul 2020
Posts: 8
|
Conversion to and from Numpy
So I'm trying to use the bm3d plugin that Vapoursynth has. Right now, I have the image as a numpy array. The bm3d plugin requires a clip as an input. Is there a way to convert a numpy array to a clip and then back to a numpy array after bm3d?
Code:
from vapoursynth import core from skimage import io import mvsfunc as mvf import numpy as np I = io.imread('D:/Jia Lab/ACsN/Python/Python Test Images/TIRF_01_10ms.tif') I = np.array(I[0]) # ERRORS flt = mvf.BM3D(I[0], sigma=3.0, profile1="fast") |
![]() |
![]() |
![]() |
#2 | Link |
Professional Code Monkey
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,497
|
1. Load images with imwri.Read() and that end of the problem should be solved
2. After the filter do "flt.get_frame(0).get_read_array(<plane number here>)" That will return numpy usable views of the frame.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet |
![]() |
![]() |
![]() |
#3 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
something:
Code:
from vapoursynth import core import mvsfunc as mvf import numpy as np clip = core.imwri.Read('D:/Jia Lab/ACsN/Python/Python Test Images/TIRF_01_10ms.tif') #one frame clip flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") npArray = np.dstack([np.asarray(flt.get_frame(0).get_read_array(i)) for i in range(3)]) #print(npArray) Last edited by _Al_; 20th July 2020 at 21:22. |
![]() |
![]() |
![]() |
#4 | Link |
Registered User
Join Date: Jul 2020
Posts: 8
|
Unlike in the test example I posted, I did some preprocessing on the image after I read it in using the numpy library. Is there a way to convert the numpy array into a clip? I know I could probably save the numpy array as a tiff file and then read it in again with imwri but I'm trying to preserve some speed. Any options?
|
![]() |
![]() |
![]() |
#5 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
you can do something like this, using ModifyFrame, which needs a placeholder vs clip with dimension as your numpy and guessing it is uint8 (RGB24 in vapoursynth, 8bit):
Code:
import vapoursynth as vs from vapoursynth import core import mvsfunc as mvf import numpy as np import functools #you have your npArray here, npArray=I clip_placeholder = core.std.BlankClip(width = npArray.shape[1], height = npArray.shape[0], format= vs.RGB24, length=1) def get_vsFrame(n, f, npArray): vsFrame = f.copy() [np.copyto(np.asarray(vsFrame.get_write_array(i)), npArray[:, :, i]) for i in range(3)] return vsFrame clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_vsFrame, npArray=npArray)) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") npArray = np.dstack([np.asarray(flt.get_frame(0).get_read_array(i)) for i in range(3)]) Last edited by _Al_; 20th July 2020 at 22:33. |
![]() |
![]() |
![]() |
#6 | Link | |
Registered User
Join Date: Jul 2020
Posts: 8
|
So I did this and got an error on the last line. I've posted the error below. I think the problem lies within flt.get_frame(0) but not sure how to resolve it. The clip description before and after bm3d is also shown.
Quote:
|
|
![]() |
![]() |
![]() |
#7 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
You are using I[0], just a part of it, instead of whole array I. But for RGB numpy array you need all three planes.
But isn't skimage format in numpy already? This should be enough: Code:
npArray = io.imread('D:/Jia Lab/ACsN/Python/Python Test Images/TIRF_01_10ms.tif') for i in range(3): or for i in [0,1,2]: does not matter, but that error was caused using that I[0] |
![]() |
![]() |
![]() |
#8 | Link | |
Registered User
Join Date: Jul 2020
Posts: 8
|
Makes sense. Made that change. Now using two frames. Code is below. Getting a new error now though. I tried changing the length parameter from 1 to 2 but the error still persists (pasted below code). Just for reference, I transposed the original numpy array to get a shape of (256, 256, 2) instead of (2, 256, 256). If I don't to that the clip ends up having a height of 2 and nothing prints out when I try to print the shape of the numpy array after dstack.
Code:
I = io.imread('D:/Jia Lab/ACsN/Python/Python Test Images/TIRF_01_10ms.tif') npArray= np.array(I[0:2]) npArray = np.transpose(npArray).copy() ## to get the shape (256, 256, 2) instead of (2, 256, 256) clip_placeholder = core.std.BlankClip(width = npArray.shape[1], height = npArray.shape[0], format= vs.RGB24, length=2) print(clip_placeholder) def get_vsFrame(n, f, npArray): vsFrame = f.copy() [np.copyto(np.asarray(vsFrame.get_write_array(i)), npArray[:, :, i]) for i in range(3)] return vsFrame clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_vsFrame, npArray=npArray)) print(clip) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") new = np.dstack([np.asarray(flt.get_frame(0).get_read_array(i)) for i in range(2)]) Quote:
|
|
![]() |
![]() |
![]() |
#10 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
If its grayscale, then you do not need three planes, just one plane only, using shape like this for example:
Code:
[[169 168 167 ... 114 117 106] [166 168 168 ... 117 116 117] [168 170 169 ... 116 120 118] ... [ 43 55 124 ... 142 140 138] [ 45 44 41 ... 149 148 152] [ 39 43 44 ... 168 169 169]] Code:
#you have your npArray here, shape it accordingly clip_placeholder = core.std.BlankClip(width = npArray.shape[1], height = npArray.shape[0], format= vs.GRAY8, length=1) def get_clip(n, f, npArray): vsFrame = f.copy() [np.copyto( np.asarray(vsFrame.get_write_array(0)), npArray )] return vsFrame clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_clip, npArray=npArray)) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") npArray = np.asarray(flt.get_frame(0).get_read_array(0)) |
![]() |
![]() |
![]() |
#12 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
ok, tried it again changing ModifyFrame calling "get_vsFrame" not "get_clip" and it works, perhaps some name conflict,
also changed it to GRAY16: so input npArray is shaped like this, 16bit integers: Code:
[[43360 43011 42745 ... 29113 29930 27196] [42587 43012 42965 ... 29834 29803 29883] [42949 43469 43358 ... 29583 30636 30165] ... [11099 14036 31621 ... 36331 35814 35311] [11492 11147 10472 ... 38043 37966 38878] [10095 10958 11336 ... 43084 43227 43198]] Code:
import vapoursynth as vs from vapoursynth import core import mvsfunc as mvf import numpy as np import functools #you have your npArray input here clip_placeholder = core.std.BlankClip(width = npArray.shape[1], height = npArray.shape[0], format= vs.GRAY16, length=1) def get_vsFrame(n, f, npArray): vsFrame = f.copy() np.copyto( np.asarray(vsFrame.get_write_array(0)), npArray ) return vsFrame clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_vsFrame, npArray=npArray)) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") npArray = np.asarray(flt.get_frame(0).get_read_array(0)) #flt.set_output() Last edited by _Al_; 23rd July 2020 at 18:10. |
![]() |
![]() |
![]() |
#14 | Link |
Registered User
Join Date: Jul 2020
Posts: 8
|
As a follow up to this, what if I wanted to do the same thing with multiple frames. So instead of the input being (256, 256, 1), the input numpy array is (256, 256, 10). So 10 frames in this case. Tried tinkering around with the code but I kept getting some array index problems. Thanks in advance.
|
![]() |
![]() |
![]() |
#15 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
You can try to loop that line with ModifyFrame:
Code:
for i in range(0,10): clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_vsFrame, npArray=npArray[:, :, i])) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") new_npArray = np.asarray(flt.get_frame(0).get_read_array(0)) #here you add your new_npArray into your new_stack_npArray, whatever you do it Last edited by _Al_; 29th July 2020 at 00:59. |
![]() |
![]() |
![]() |
#16 | Link |
Registered User
Join Date: Jul 2020
Posts: 8
|
I would do this but I specifically want to use the VBM3D functionality that mvf.BM3D provides. So the input clip should be multiple frames. I guess the question is if there is a way to make a (256, 256, 10) numpy array into a 10-frame clip and then back to an array? That way I can use VBM3D instead of denoising each frame separately through a loop.
|
![]() |
![]() |
![]() |
#17 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
yes, try this:
Code:
clip_placeholder = core.std.BlankClip(width = npArray.shape[1], height = npArray.shape[0], format= vs.GRAY16, length=10) def get_vsFrame(n, f, npArray): vsFrame = f.copy() np.copyto( np.asarray(vsFrame.get_write_array(0)), npArray[:, :, n] ) return vsFrame clip = core.std.ModifyFrame(clip_placeholder, clip_placeholder, functools.partial(get_vsFrame, npArray=npArray)) flt = mvf.BM3D(clip, sigma=3.0, profile1="fast") something like: Code:
new_arrays = [] for i in range(0,10): new_arrays.append(np.asarray(flt.get_frame(i).get_read_array(0))) npArrays = np.dstack(new_arrays) del new_arrays Code:
npArrays = np.dstack([np.asarray(flt.get_frame(i).get_read_array(0)) for i in range(10)]) or if final shape of npArrays was wrong try to stack it as you do, you get a general idea how to get one frame array: np.asarray(flt.get_frame(i).get_read_array(0)) Last edited by _Al_; 29th July 2020 at 23:01. |
![]() |
![]() |
![]() |
#19 | Link |
Registered User
Join Date: May 2011
Posts: 273
|
R55 and above planes are now accessible thru slicing python feature, so you'd use:
Code:
np.array(f[p]) Code:
API4 = True try: core.std.BlankClip().get_frame(0)[0] except: API4 = False . . . f = rgb_clip.get_frame(0) if API4: img = np.dstack([np.asarray(f[p]) for p in [2,1,0]]) else: img = np.dstack([np.asarray(f.get_read_array(p)) for p in [2,1,0]]) |
![]() |
![]() |
![]() |
Tags |
clip, numpy, python |
Thread Tools | Search this Thread |
Display Modes | |
|
|