Hi
How shoud I use Enhancer software's VA library together with avcodec2 library ? I was following the examples. Do I really need to create 4 different buffers ? I noticed avcodec library supports VAAPI. AVCodecContext has hwaccel_context which seems to be null so I'll have to alloc memory for it by myself and use it as struct vaapi_context. But which function should I call that avcodec would fill up that struct vaapi_context (some of the fields have to be filled by the user and some of them should be filled by avcodec). Also vaapi_context has slice params and slice data fields but not picture params or iq matrix. Where should I find those two from ?
Trying to decode some video I usually get ring 3 being busy and GPU being busy messages when calling vaPutSurface. Or vaRenderPicture fails with some settings.
Using hardware video decoding with avcodec v2 is very obscure. For DVPlayer I had to figure it out from the info in this link, and by analysing the avcodec v2 source-code.
You basically need to set a custom get_format hook that will check if the format can be hardware accelerated, and set up the hwaccel codec context field. The get_format hook also needs to check if hwaccel was already set, because avcodec may call it again later. If everything is set up correctly, then when a frame has been decoded, you can use vaPutSurface() with videoContext-:hwaccel_context to draw the frame.
One other really weird thing, is that the VASurface is in AVFrame->buff[3]. This is hard-coded, and there's no API call to get it in a friendly way.
I never did finish my own code to use it, so I don't have a working example. Contact me via email, and I can send you the code that I have.
The latest avcodec API is easier to use, but we don't have that yet...
Hans
Thanks!
Now I'm at that point when trying to create the surface it gives an error: "Failed to create VAAPI surface: 3 (invalid VADisplay)".
If I'll try to use vaDisplay directly instead of the one in hwaccel structure when creating the surface it complains: "Failed to create VAAPI surface: 14 (the requested RT Format is not supported)."
@TSK I suggest you look at the MPEG decoding example in the VALib SDK. It's been a while since I worked on it, and I can't remember the details.