Latest post Wed, Dec 15 2021 3:10 PM by Jeroen van Eekeres. 13 replies.
Page 1 of 1 (14 items)
Sort Posts: Previous Next
  • Wed, Dec 1 2021 1:59 PM

    • nicov
    • Top 500 Contributor
    • Joined on Wed, Jun 7 2006
    • Posts 224
    • Points 2,930

    dongle free distributed processing

    It would be great to be able to use a second computer processing power to do renders and transcodes on jobs that don't justify nexis, media central and collaborative tools that today are a prerequisite to distributed processing. The price of avid shared storage is high and hardly justified for many but a lot of small shops and freelancers have multiple computers and could benefit from some kind of distributed processing. Background rendering and transcoding has its limits, it just slows down operations and playback and takes a lot of time to bring your renders and transcodes online.

    I'm on mac and recently made the move to a new macbookpro, I'm keeping a capable i9 imac for the transition period as some plugs and softwares like MC do not support the new chip yet. It would be great if in time we could use the imac for processing while editing on MC with the new macs. Maybe some cripled version of distributed processing that only allows one extra computer and with support for an affordable third party shared storage solution. With postlab maybe, that would even bring to possibility to work remote and still benefit from the processing.. 

    MacOS 12.4 / MC 2022.7 / 16'MacBookpro M1max 64gb / BMD ultrastudio monitor3G > JVCDTV24 / tangent wave2/ TB xtrm-q sabrent nvme ssd 8tb [view my complete system specs]
  • Wed, Dec 1 2021 10:24 PM In reply to

    • Adman
    • Top 25 Contributor
    • Joined on Thu, Oct 13 2005
    • Atlanta Georgia, USA
    • Posts 5,487
    • Points 62,205
    • BlogAuthor
      Moderator: Avid Free DV
      Moderator: Avid Xpress DV PC
      Moderator: Avid Xpress Studio
      Moderator: MCA PC

    Re: dongle free distributed processing

    Good idea.

    https://www.avid.com/products/media-composer-distributed-processing

    "Stay in your creative video editing flow by offloading processor-intensive media tasks to any unused computers"

    Mac Pro, Monterey, 64GB Ram. 1TB SSD system drive, 250Gb SSD external (media) drive. QHD Ultra wide, thunderbolt LG monitor, MC 2021.12 Pro Tools 2021... [view my complete system specs]
  • Thu, Dec 2 2021 4:46 PM In reply to

    Re: dongle free distributed processing

    Adman:

    Good idea.

    https://www.avid.com/products/media-composer-distributed-processing

    "Stay in your creative video editing flow by offloading processor-intensive media tasks to any unused computers"

    I was so excited when I saw that... until I read that you needed lots of infrastructure to support that system. 

    I would love to see this, my sidestation could be transcoding rushes for my next project, while I'm editing another one. Yeah I know I can do that in Media Encoder, but I don't want to deal with the faff of importing media through databases. 

    Editing Movie Magic.

    My Equipment & System Specs

    Media Composer Ultimate 2022.7 | Pro Tools Studio 2022.7 | Avid Play | Sibelius Artist 2022.7

  • Fri, Dec 3 2021 10:33 PM In reply to

    Re: dongle free distributed processing

    Philip Kapadia:
    I would love to see this, my sidestation could be transcoding rushes for my next project, while I'm editing another one. Yeah I know I can do that in Media Encoder, but I don't want to deal with the faff of importing media through databases. 
    I do my media input using a second machine. See workflow described in this thread.    https://community.avid.com/forums/t/183202.aspx   

    This described it for me using 1 computer with  both programs on the same machine. With 2 or more machines you can replicate the process using USB3 portable drives or network to move the numeric numbered media folders across to you edit computers Media drive Avid Media Files folder. No importing media as Avid finds and indexes this folder. (20 seconds worst case scenario). Then drag the resultant msmMMOB.mdb into your desired bin. It then populates it with your media virtually instantly.

    This works with either Resolve (free) and Resolve Studio.  This background process saves me dozens of hours each year.

  • Wed, Dec 8 2021 10:13 AM In reply to

    • nicov
    • Top 500 Contributor
    • Joined on Wed, Jun 7 2006
    • Posts 224
    • Points 2,930

    Re: dongle free distributed processing

    Philip Kapadia:

    I was so excited when I saw that... until I read that you needed lots of infrastructure to support that system. 

    I would love to see this, my sidestation could be transcoding rushes for my next project, while I'm editing another one. Yeah I know I can do that in Media Encoder, but I don't want to deal with the faff of importing media through databases. 

    That's precisely the point. It would be great to have a way of using somekind of distributed processing without nexis. Transcodes of new material coming in mid projet or long renders would no longer stop the editing process (or slow down the system with background tasks).

    I understand the need of a shared storage solution for distributed processing to work but since you can "Enable Bin Sharing on 3rd party storage emulating Avid NEXIS/ISIS" why locking the distributed processing feature to proprietary storage?

    This could even work without a NAS. We have two macs already connected thru thunderblot bridge so storage is network mounted and shared on both computers (with afp or smb protocol choice).

    I guess the alternative for now is adding a second MC license plus Mimiq and run MC on both computers working on a shared project and using one MC for those jobs and the other one for actual editing but that's more cumberstone when working solo, having to switch from one computer to the other.. Plus the cost.

    Hence my feature request, distributed processing for all, just two computers needed. And maybe as a limitation to preserve nexis just one client allowed so you cannot add multiple computers and build a render farm.

     

    MacOS 12.4 / MC 2022.7 / 16'MacBookpro M1max 64gb / BMD ultrastudio monitor3G > JVCDTV24 / tangent wave2/ TB xtrm-q sabrent nvme ssd 8tb [view my complete system specs]
  • Wed, Dec 8 2021 2:59 PM In reply to

    Re: dongle free distributed processing

    That there is a market for 'small shop distributed processing' is obvious for anyone that works for more than a few days in this business but ask yourself this:

    If you manufactured Nexis, Mediacentral Production (Interplay) and Mediacentral cloud UX. for whom would you create this feature first? And after doing so would you create the option to do the same thing without spending a lot $$$ on the above?

     

    From the old Apple Quadro 950 to HP Z8xx. My current own systems: 1x Z420 E5 1650 32GB memory quadro K2200, 1x XW8600, 2x 3.0Ghz Quadcore, 24GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

  • Wed, Dec 8 2021 3:48 PM In reply to

    • nicov
    • Top 500 Contributor
    • Joined on Wed, Jun 7 2006
    • Posts 224
    • Points 2,930

    Re: dongle free distributed processing

    Jeroen van Eekeres:

    That there is a market for 'small shop distributed processing' is obvious for anyone that works for more than a few days in this business but ask yourself this:

    If you manufactured Nexis, Mediacentral Production (Interplay) and Mediacentral cloud UX. for whom would you create this feature first? And after doing so would you create the option to do the same thing without spending a lot $$ on the above?

     

    Sure but things change and eventually get more open, that's the way it's always been. We did get openIO after decades of proprietary hardware, and symphony thrown in media composer to name just 2...

    I could live with it being a paid add-on and having some limitations if it would speed up my workflow.

    MacOS 12.4 / MC 2022.7 / 16'MacBookpro M1max 64gb / BMD ultrastudio monitor3G > JVCDTV24 / tangent wave2/ TB xtrm-q sabrent nvme ssd 8tb [view my complete system specs]
  • Wed, Dec 8 2021 5:29 PM In reply to

    Re: dongle free distributed processing

    I'm sure I'm missing something but I don't understand why these useful features get locked away from your average user. They're doing the same with the SRT stuff too. It's not like enterprise users won't use that stuff if it's available to the proles who don't own a Nexis.

    I'm working from home now and haven't been in a facility for a while but I have two machines here and would love to network them together and use the distributed render feature. I do a lot of exports and I do a lot of very long renders due to extensive noise reduction and if I could shunt those off to the second machine that would be a big time saver. 

    Avid gets stuck in that old mindset of add-ons and and special features but the truth of it is that they should just be making the best product they can and including everything for everyone whether enterprise or one man band. It's what Resolve is doing - they don't hold out high end features - you pay £300 and you get the lot. 

    Unless there is some technical reason for locking us out I think they should be opening up these features to the rest of us who pay our subs. People aren't going to not buy a Nexis or Interplay server because they could just fudge it with off the shelf hardware, those enterprise customers will buy that regardless. But by making it available to all, now mid sized and smaller places that may not be buying a Nexis can put together their own setup and surely the more people who have seats of MC the better?  Enterprise customers are gonna enterprise regardless. Everyone should have access to the full set of features, it only serves to make MC a more compelling offering in an NLE market that is giving it some stiff competition. 

    Andi

    HP ZBook G6 15 2.6Ghz 6-core i7 9th Gen Nvidia Quadro T2000 4GB 64GB RAM Win 10 Pro MC Ultimate 2022.10 MacbookPro 15 2019 6 core 2.6GHz i9 32GB... [view my complete system specs]
  • Thu, Dec 9 2021 9:04 AM In reply to

    Re: dongle free distributed processing

    ripvanmarlowe:
    Everyone should have access to the full set of features,

    While we all would love that... we also understand that the above has to say: Everyone WHO PAYS for it should have access to the full set of features. Askiing Avid to give away features integrated with their own products that costed $$$ to develop over the past XX years and that were sold for more $$$ to make a profit, makes little sense.

    Of course technology gets cheaper while performing better over time and I fully agree that limiting some features, like SRT, to enterprise licenses is strange to say the least. But distributed processing will always require shared storage and obviously a layer to manage distribution. You want to be able to manage which machines participate in the distribution and, depending on the performance of each machine, regulate which job is performed by which machine. It wouldn't be the first time where adding slower machines, processor, GPU, memory, network connection, to an existing render farm actually slows down render times.

     

     

    From the old Apple Quadro 950 to HP Z8xx. My current own systems: 1x Z420 E5 1650 32GB memory quadro K2200, 1x XW8600, 2x 3.0Ghz Quadcore, 24GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

  • Thu, Dec 9 2021 12:35 PM In reply to

    Re: dongle free distributed processing

    Jeroen van Eekeres:
    Everyone WHO PAYS for it should have access to the full set of features.

    I am a paying customer last time I checked. I'm not asking them to give away features - I'm literally paying for that.

    Jeroen van Eekeres:
    You want to be able to manage which machines participate in the distribution and, depending on the performance of each machine, regulate which job is performed by which machine.

    Isn't this what the feature does? From Avid's Distributed Processing website page: 

    "Maximize your resources

    Put all of your idle, unused, or older computers, servers, and workstations to work by networking them together as a powerful media render and transcode farm, increasing your ROI.". So this sounds like adding an older machine is an option so I presume they've optimised the system to not slow down when you add mismatching hardware? My point is, I am paying a fee for a program and I am having features locked away from me, I don't even have the option of paying extra for this feature. As a small business I would also like to increase my ROI but I can't despite the fact that it would cost Avid nothing to allow me to use this feature. As I said before, the enterprise users are going to be buying their Nexis anyhow, so Avid is not losing revenue by giving me access to that feature. What I might do is buy another seat of MC because now I can have a small render farm and Avid has made a bit of extra profit from a user that otherwise wouldn't have considered doing that. They still make their Enterprise cheese don't they? Maybe someone who businesses better than me can explain the downside to keeping this an enterprise only option. Andi
    HP ZBook G6 15 2.6Ghz 6-core i7 9th Gen Nvidia Quadro T2000 4GB 64GB RAM Win 10 Pro MC Ultimate 2022.10 MacbookPro 15 2019 6 core 2.6GHz i9 32GB... [view my complete system specs]
  • Thu, Dec 9 2021 5:21 PM In reply to

    Re: dongle free distributed processing

    ripvanmarlowe:
    I am paying a fee for a program and I am having features locked away from me

    I don't know if you have ever used media composer on a site where it is integrated in mediacentral production and/or mediacentral cloud UX. If you want I can give you a quick look around on one system although it will not be using distributed processing.

    The point of it would be to give you a quick insight on the role of mediacentral production, Interplay and its media indexer, on Nexis. When you understand that and what a production services engine is and which functional parts of these servers is also present in media composer for background services, I'm pretty sure you would see my side of the argument.

    I understand your argument. I consider it valid for a very large part but if you understand that some of these components have approx. 20 years of development history behind them and Avid keeps maintaning backwards compatibility from Interplay version 1 to 2, to 3, to 2017.xx to 2018.xx, etc... up to what is now called mediacentral production 2021.xx, you will probably also understand why Avid continues this:

    ripvanmarlowe:
    mindset of add-ons and and special features

    Simply said. I hope Avid will eventually develop mediacentral editorial into a mediacentral that has distributed processing included instead of needing 5-8 different servers each needing hardware and licenses and complex setup. But also in this case an Avid shared storage will be needed. 

    From the old Apple Quadro 950 to HP Z8xx. My current own systems: 1x Z420 E5 1650 32GB memory quadro K2200, 1x XW8600, 2x 3.0Ghz Quadcore, 24GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

  • Wed, Dec 15 2021 9:53 AM In reply to

    • nicov
    • Top 500 Contributor
    • Joined on Wed, Jun 7 2006
    • Posts 224
    • Points 2,930

    Re: dongle free distributed processing

    I was just suggesting that the code behind distributed processing could be tweaked to offer a feature for people working with 3rd party shared storage and having two computers. Forget about mediacentral and the ability to set up a render farm. Just a way to transfer transcodes and render tasks to one other computer sitting nearby to preserve ressources on the edit bay for real-time playback, trim and other operations. Call it "sidekick" or whatever marketing feels like instead of "distributed processing". I have an IMac that would do great for that purpose, no worries it won't end up slowing the process, in fact MC rarely uses it's processing capabilities above 80-90% even when transcoding to dnxhr..

    I think it would just be a great feature and would add value to the package.

    MacOS 12.4 / MC 2022.7 / 16'MacBookpro M1max 64gb / BMD ultrastudio monitor3G > JVCDTV24 / tangent wave2/ TB xtrm-q sabrent nvme ssd 8tb [view my complete system specs]
  • Wed, Dec 15 2021 11:46 AM In reply to

    Re: dongle free distributed processing

    Having had this setup previously with Avids DS its not something I'd pursue.

    It wouldn't make sense to tie a second machine to the main edit machine. Facilities with more than one edit bay would moan they can't share that render system. So it would have to be a network node that can be accessed by multiple machines and handle those jobs passing the results back to the correct systems.

    The overhead of deperating these render tasks from foreground processing and then farming them out takes time (and resources) you then have to have a way of recieving the resultant renders back and integrating them and upodating databases etc.

    In my experience the delay in that round trip was a pain and editors frequently just wanted the resuklt then and there so did it as a foreground render for speed.

    But with modern multiple CPU multiple core processors and loads of RAM it just makes more sense to extend the background processing that we have already in place. Lets just see more render tasks with the option to send to the background.

    And work on the background render processes to make smarter use of the onboard resources.

    HP Z840 3.1GHZ 20cores 128GB RAM M4000 GPU 1TB NVMe drive HP Z book 17 G2 2.7GHZ Quad core 32GB RAM Nvidia K3100M 1TB SSD drive ACI Moderator. I'm... [view my complete system specs]

     

    Broadcast & Post Production Consultant / Trainer  Avid Certified Instructor VET (Retired Early 2022)

     

    Still offering training and support for: QC/QAR Training - Understanding Digital Media - Advanced Files * Compression - Avid Ingest - PSE fixing courses and more.

    Mainly delivered remotely via zoom but onsite possible.

     

    T 07581 201248 | E pat@vet-training.co.uk

     

  • Wed, Dec 15 2021 3:10 PM In reply to

    Re: dongle free distributed processing

    nicov:
    I think it would just be a great feature and would add value to the package

    And I think no Avid Media composer customer will disagree with you. I don't.

    All I'm trying to say is that this feature is currently part of the >$ 250.000,- systems at Avid. I hope it becomes a feature of the +/- $ 30.000,- and up systems. Asking for it to be included in the $ 49,- a month media composer rental would devalue the Nexis and all Mediacentral products. Making it part of a Mediacentral editorial management with Nexis Pro bundle, that makes sense to me.

     

    From the old Apple Quadro 950 to HP Z8xx. My current own systems: 1x Z420 E5 1650 32GB memory quadro K2200, 1x XW8600, 2x 3.0Ghz Quadcore, 24GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

Page 1 of 1 (14 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller