Latest post Fri, Sep 5 2014 8:45 PM by mdombros. 20 replies.
Page 1 of 2 (21 items) 1 2 Next >
Sort Posts: Previous Next
  • Wed, Oct 2 2013 6:50 PM

    • ranimmo
    • Not Ranked
    • Joined on Sat, Jul 27 2013
    • Posts 7
    • Points 85

    Transcode Error "Upstream pipe stall"

    Okay, I am new to avid, and I am about ten seconds away from asking for my money back and editing on premiere.  I have spent two days now trying to get this software to work and I haven't even gotten any clips into a timeline yet.  This is ridiculous.

    I am trying to transcode 165 F65 RAW files to DNxHD proxies.  I have all the files AMA linked and they work great.  But they are 4K RAW files.  So playback is not the best.  I am trying to transcode DNxHD Proxies and everytime I do I get the following error message:

    "Exception: NEMFileWriter:: ExecuteInternal TIMEOUT - NEMFileWriter has detected upstream pipe stall..."

    Great.  Whatever that means.  I can't seem to find any answers on how to solve this.  It isn't because I have too many files, I only 165. I have the correct Nvidia drivers that come with Media Composer.  There is plenty of space on the destination drive.  I have tried to send the files to my 3TB internal RAID, to my 1TB SSD OS Drive, and to my 2TB Internal Media Drive.  None of them work.  I have tried rebooting, reinstalling.  I have tried all the various DNxHD compression rates.  None work.  At one point it managed to transcode five files before dumping out again.

    What the *** avid?  Can someone please tell me what this error message means and how I can fix it?

     

    Here are my computer specs:

    2x Intel Xeon E5-2640 Six Core Processors

    32GB RAM

    Nvidia Quadro 4000

    Nvidia Quadro K2000

    Windows 7 Ultimate SP1

    Media Composer 7.0.2

     

     

    Does anyone have any idea what would be causing this?

  • Thu, Oct 3 2013 9:54 AM In reply to

    • Vilem
    • Top 75 Contributor
    • Joined on Sat, Apr 10 2010
    • Prague
    • Posts 1,399
    • Points 15,405

    Re: Transcode Error "Upstream pipe stall"

    Verify that you have sufficient cooling PC case. I had similar problems and the stress tests the CPU and graphics card (Q4000 *** warm) I am greatly enhanced cooling and it helped.
    V.

    AVID MC2018.8 - dongle, HP Z620 - 2 x Intel Xeon E5-2620, Quadro P2000 - driver 390.77, RAM 48GB - DDR3, BM Intensity Pro 4k - 10.11, W10 - 64bit Professional... [view my complete system specs]
  • Thu, Oct 3 2013 3:33 PM In reply to

    • ranimmo
    • Not Ranked
    • Joined on Sat, Jul 27 2013
    • Posts 7
    • Points 85

    Re: Transcode Error "Upstream pipe stall"

    That is a good point, but, I have Main Gear's Quatum Shift 4K chasis with Liquid Cooling and their Performance Cooling upgrade.  http://www.maingear.com/epic/cpu.php


    I
     would howpe that is suficient for cooling.  I have had no heat problems before, even when running after effects, and premiere simultaneously, and really taxing the system.

  • Thu, Oct 3 2013 3:50 PM In reply to

    • Vilem
    • Top 75 Contributor
    • Joined on Sat, Apr 10 2010
    • Prague
    • Posts 1,399
    • Points 15,405

    Re: Transcode Error "Upstream pipe stall"

    Ok.

    Cooling looks good.


    I've had problems specifically in transcoding H264 to DNxHD. There was probably some overheating cores on the processor (maybe wrongly deposited conductive paste). Never it will not affect the render effects only when
    transcoding - MC6 really loaded with all processor cores.

    Also, I tried to test the RAM.


    V.

    AVID MC2018.8 - dongle, HP Z620 - 2 x Intel Xeon E5-2620, Quadro P2000 - driver 390.77, RAM 48GB - DDR3, BM Intensity Pro 4k - 10.11, W10 - 64bit Professional... [view my complete system specs]
  • Fri, Oct 4 2013 7:22 AM In reply to

    • newyankee
    • Top 500 Contributor
    • Joined on Sun, Sep 16 2007
    • Palo Alto, Ca
    • Posts 208
    • Points 2,630

    Re: Transcode Error "Upstream pipe stall"

    Sure seems like your system should work fine.

    Just FYI, I'll describe a situation where I got the same error. Was testing a new system and was trying to transcode AVCHD on a USB3 memory card to an 80% full single internal data drive. After some research it seemed to me that the problem was the slow data drive could not keep up with the output generated by Avid. My solution was to use a USB2 memory card reader, which in turn slowed down the input data stream, and then the data drive could keep up.

    Asus P9x79Pro MB, Intel I7 3930K overclocked to 4GHz, Quadro FX3800, MC 8.3.1 /Symphony, Raid 0 / 3ware controller, Win 7 Pro 64, 32 Gig Ram, Matrox MX02... [view my complete system specs]
  • Fri, Oct 4 2013 8:22 AM In reply to

    Re: Transcode Error "Upstream pipe stall"

    Hi

    The upstream pipestall error is something that has plagued me ever sincee AMA was introduced. It was a much bigger problem when transcoding spanned clips in my case. BUT since 7.0.2 and new ama plugins Im without any issues, so it seems that AVID have addressed it somewhat since 6.x.

    I did read somewhere online about loading the cpu with work to slow down the transcoding data rate to a speed that the drives could handle. i did this to reasonable effect by running long youtube clips, and vimeo clips whilst transcoding the spanned media. it wasnt 100% better but did improve it.

    Peter

     

    Windows 7 Pro 64, Intel i7 3.4 ghz cpu, 32 GIG DDR3, MC 8.4.5 Pro Tools 10 & 11 [view my complete system specs]

    Peter Groom

    Dubbing Mixer

  • Fri, Oct 4 2013 2:43 PM In reply to

    • ranimmo
    • Not Ranked
    • Joined on Sat, Jul 27 2013
    • Posts 7
    • Points 85

    Re: Transcode Error "Upstream pipe stall"

    Wow.  Never in my life have I heard of a solution that involves SLOWING DOWN your computer to get something done.  Haha.  Good job Avid.  That is some great programming you have there.  So my thinking was totally backwards.  Instead of turning all background programs off, I need to turn them all on.  Haha.  This is ridiculous.

    Okay, I will try having after effects render something absurdly long that is running off my OS drive to see if I can slow down my CPU to a speed my media drives can handle for Avid to transcode.  Thanks Guys.  I will let you know if this works.

  • Sat, Oct 5 2013 4:43 PM In reply to

    • ranimmo
    • Not Ranked
    • Joined on Sat, Jul 27 2013
    • Posts 7
    • Points 85

    Re: Transcode Error "Upstream pipe stall"

    So for anyone else having this problem, after much trouble shooting, I have discovered the source.  My internal media drive is bad. I am only getting 5MB/s read speed off of it.  After copying one file to my internal RAID just to test, everything worked like a charm.  It could even playback the AMA linked RAW 4K reasonably well.  Now begins the long process of copying all the data to the internal RAID with that terrible read speed bottlenecking it.

    Thanks to everyone who commented.  After reading tons of posts on this problem I suppose that this error message generally means that you have some sort of hardware issue creating a bottleneck somewhere.  For any future people searching this problem, start troubleshooting all your hardware!

  • Wed, Oct 30 2013 12:07 AM In reply to

    • fjfield
    • Not Ranked
    • Joined on Tue, Oct 29 2013
    • Posts 1
    • Points 15

    Re: Transcode Error "Upstream pipe stall"

    I am new to this forum and I had the same issue with MC 7. I was able to fix the problem, for me at least, making the following 2 changes.

    I revied the the Windows 7 Optimization and Troubleshooting here

    http://avid.force.com/pkb/articles/en_US/Troubleshooting/en349411?retURL=%2Fpkb%2Farticles%2Fen_US%2Fhow_to%2FHow-to-enroll-in-AVID-training%3Fpopup%3Dtrue%26NewLang%3Den%26DocType%3D1080%26q%3David%2Btraining%26site%3Dkbase%26filter%3D0%26subfilters%3D%26ProductCategory%3D&popup=true

    Specificially I made sure I did the following:

    Adjust Processor Scheduling

    This optimization allows the computer to run Pro Tools more efficiently.

    • Click on the Start Menu, right-click on Computer.
    • Choose Properties.
    • Choose "Advanced System Settings."
    • Under the Performance section, click the Settings button.
    • In the Performance Options window, click the Advanced tab.
    • Under the Processor Scheduling section, select the Background Services option.
    • Hit Apply and then OK to close the Performance Options window.
    • Click OK to close the System Properties window.
    • Restart the computer for the changes to take effect.

    Additionaly, I plugged in my card reader, with the offending very long clip, in the USB (3.0) slot before I starded Media Composer. When I started MC and then openned the project all my clips on the card reader (from my camcorder) openned in a new bin and I simply dragged the clip I wanted into the bin I needed it in and MC automatically imported the clip. It worked without any issues. I think I will be doing this from now on. Much easier than using the import method. Saves hard drive space. 

    Hope this helps someone else. 

  • Wed, Feb 19 2014 4:41 PM In reply to

    • LKJ JKL
    • Not Ranked
    • Joined on Tue, Jun 2 2009
    • Posts 120
    • Points 1,405

    Re: Transcode Error "Upstream pipe stall"

    here is what worked in my enviroment (z420, win7, MC 703):

     

    I got the error while transcoding clips from a canon xf105.

     

    What I  did:

    > clips are on an external SSD connected via usb 3 to the z420

    > bring clips via ama into the avid

    > consolidate to dnx185 onto the internal raid0 HDDs

    > get the Upsteam-Stall-Error (a lot of times!)

     

    solution:

    > connect the ext SSD to an usb 2 port - all works fine now, with tons of clips!

     

    good luck!

    HP Z420 (E5-1650v2) | 32GB RAM | 250 GB SSD (OS) | NVIDIA Quadro P2000 | internal raid0 | AJA Io Express | Win7 Pro64 Sp1 | MC v8.10 [view my complete system specs]
  • Wed, Feb 19 2014 7:45 PM In reply to

    • mdombros
    • Not Ranked
    • Joined on Mon, Sep 30 2013
    • Posts 17
    • Points 225

    Re: Transcode Error "Upstream pipe stall"

    Ok, so forgive me but I am a database kernel engineer for the worlds largest database company and the idea that the transcodes cannot be programmed to handle a disk I/O wait is baffling to me. We do it all the time with datastreams much larger and faster than anything on a Windows PC.

    i encounter this problem CONSTANTLY importing AVCHD files. AMA link followed by transcode seems to solve the problem but is not a solution for some of my work flows. I have tried every solution on this forum, you tube videos (ludicrous), reading from slower drives etc. no solution.

    Are my drives slow? Not for any thing else I do. Yes they are kinda full and are probably not completely optimized but software can be written to handle disk I/O properly.

    This is a bug and it needs to be fixed, plain and simple. If Avid wants to work with me on this one I am more than happy to provide info and work with support or engineers to solve the problem. I work in Burlington, I could bring my machine over to you.

    MC 7.0.2, Intel core i7,  16 gb RAM, (2) WD 1 TB blue drives, Win 7 x64

    matt

  • Wed, Feb 19 2014 7:50 PM In reply to

    • mdombros
    • Not Ranked
    • Joined on Mon, Sep 30 2013
    • Posts 17
    • Points 225

    Re: Transcode Error "Upstream pipe stall"

  • Wed, May 21 2014 9:50 PM In reply to

    • SortVid
    • Not Ranked
    • Joined on Tue, Mar 27 2012
    • Posts 1
    • Points 20

    Re: Transcode Error "Upstream pipe stall"

    I am no database kernel engineer - but I am one very frustrated producer. At the moment we are loading four hours of AVCHD-material each day. We get stuck with the upstream pipe stall constantly. And actually have for the last two years. Come on guys?!? Call Matt and get this problem fixed…
     
    We have tried everything and I am telling you: The only thing that works is to slow down your computer. YouTube is a good possibility – though I prefer Vimeo. But I still need at least two windows running with HD-video.
     
    And maybe some of you found working solutions with all kinds of different setups. I am not so lucky yet. I have tried FireWire, Thunderbolt, USB2, USB3 and eSATA. I have tried several different kinds of RAID and discs. But the good old pipe stall always returns to haunt me.
     
    Maybe we shouldn’t blame Avid, but Apple for making to fast computers? Apple – slow down those computers…
     
    / Tue
     
    MacBook 2.4 GHz Core i7 with 16 Gb RAM / Avid MCP 6.0.3

    Filed under:
  • Wed, May 21 2014 10:49 PM In reply to

    • Ken984
    • Not Ranked
    • Joined on Thu, Oct 13 2005
    • Louisiana
    • Posts 88
    • Points 1,085

    Re: Transcode Error "Upstream pipe stall"

    One other suggestion I do not see in this thread. Try it with Hyperthreading off. I read somewhere on here that with 12 actual cores that Hyperthreading off seemed to help with these random errors. It is worth a shot anyway.

    DIY. Intel 3930k I7, Gigabyte GA-X79-UD3 MB, Nvidia Quadro 2000, 16GB ram. 256GB Samsung 840Pro SSD boot drive, 1.5TB SATA for internal storage, 4TB G... [view my complete system specs]
  • Thu, May 22 2014 3:11 AM In reply to

    • mdombros
    • Not Ranked
    • Joined on Mon, Sep 30 2013
    • Posts 17
    • Points 225

    Re: Transcode Error "Upstream pipe stall"

    Well, I recenty solved my problem.....I think. Was waiting for a few more tests but since you revived this thread I don't have to go looking for it :-)

    While diagnosing another disk related issue (external USB 3.0 disks) I realized I was using the Microsoft SATA driver on my machine instead of the Intel RAID driver that came with the motherboard.  From all the documentation, one would think the RAID driver really only did RAID things and since it loaded some service at startup that I didn't use I uninstalled it. Well, I found some musings on the Intel site that led me to believe the RAID driver was optimized for whatever SATA stuff the chipset did so I reinstalled it and BINGO... I have not had an upstream pipe stall since. (Knock on silicone)

    So, folks, this has nothing to do with the speed of your computer and has everything to do with the speed of your disk I/O (as I knew but am tired of hearing people say "slow down the computer"). The YouTube thing never worked for me because on my Intel I7 it never had an effect, CPU just screams. And there is just something inherently wrong with clogging your CPU to the point that it uses disk more slowly. That will also slow the transcode happening on import.

    My conclusion is, if you are getting pipe stalls you should focus on speeding up or optimizing your disk I/O. Better interface, faster drive, bigger cache, better cables, better drivers (as in my case). I wonder if disabling write caching makes it worse or if enabling would make it better....a setting to check that I don't recall anyone mentioning before.

    This in NO way excuses the SE that wrote this stuff in such a way that it DIES on an I/O wait. I have tried to get a job with Avid several times, I already work in Burlington but they haven't been willing to pay me what I am worth unfortunately :-) if you want software that works, well, you get what you pay for.  No really, love Avid....would still love to work there. Hire me I can help!

    Matt

Page 1 of 2 (21 items) 1 2 Next >

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller