How To: Render LightWave Files on the ResPower Super/Farm™

Overview

The ResPower Super/Farm™ Render Farm proudly supports LightWave rendering using either LightWave’s network rendering engine, LWSN, or with FPrime via Worley’s fantastic (if unofficial) utility, WSN.

This tutorial discusses how to use the ResPower Super/Farm™ for rendering .lws files created using LightWave 9.0 through 10.1. To find out if the ResPower Super/Farm™ supports a specific version of LightWave, simply upload your .lws file and go to the Render Wizard. If the version you want to use shows up in the list of render engines on the Render Wizard, ResPower currently supports it.

There are a few basic steps to take, which will be discussed in detail below:

  • Prepare your content directory
  • Upload your content
  • Submit your job for rendering
  • Monitor your job’s progress
  • Download the frames
  • Pay

 

First Time Users

First time users can try out the Super/Farm with zero cost. This will help you to get a better feel for what the costs are and to get through the learning curve. To take advantage of this, please follow this tutorial to submit 10 frames from a scene that is indicative of the work that you do.

When selecting your 10 frames, be sure to space them evenly throughout your scene. For example, if you expect to render a 1000 frame animation, submit every 100th frame using the “Frame Step” option on the Render Wizard. You would then multiply the Credits used by 100 to get the closest approximation to final render cost for that scene.

When submitting your job, please be sure to use the “Metered” billing model. The metered service allows you to run a job to completion, see the total cost, and then choose the “Pay For Output” action on the job status page. To run a test job, all you need to do is run the job with the metered service, and simply skip the last step of purchasing the finished output. This allows you to see the total cost for the job without downloading the finished frames and without actually paying for them. If during the course of running your job the cost becomes too high, simply abort the job on the Job Status page and re-configure your scene to take an amount of time that will meet your budget.

If you expect to use the Super/Farm to perform still frame work, please submit a single frame with 1/2 of the target width and height, using the Split Frame rendering mode. You would then multiply by 4 to get a close approximation to final render cost.

Prepare Your Content Directory

Standard LightWave Content Directory – Your scene must reside inside of a standard LightWave content directory. All textures, .lwo’s, etc., must be inside of this same directory, or in subdirectories of it. The standard layout of a LightWave content directory looks like this:

…/ content-folder-root

/ images

texture001.jpg
texture002.jpg

texture00n.jpg

/ objects

object001.lwo
object002.lwo

object00n.lwo

/ scenes

scene001.lwo
scene002.lwo

scene00n.lwo

Furthermore, they must be added to the scene while LightWave has been configured to use that content directory via the “Edit -> Set Content Directory… *F12″ option. If they were added prior to this, they may be mislinked, leading to failure at render time.

 

Configure Camera and Render Settings – Take special care to set antialiasing, the “ray recursion limit”, and Global Illumination parameters in a manner that will yield quality and render times you can live with.

Missing resources – If you have any missing resources, the scene will fail to render. This includes textures, plug-ins, etc. If you cannot remove references to resources that are not in use, be sure to upload them to the farm. For plug-ins, you will need to contact support staff to have them installed. If you do not have a texture and you know that it is not necessary to render the scene, create a blank texture with the same name.

Mislinked textures – If you have any mislinked textures, the scene may fail to render. A mislinked texture is one that is linked in using a hard coded path, even though it is in your project directory. This is typically caused by a texture that was linked in while LightWave did not have its content directory set properly.

Special characters in filenames – Any .lws,.lwo or texture file that uses a special character in a file name will fail to work properly on the Super/Farm™. The following characters are the only supported characters for filenames:

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_-.

For example, a file named

¾view.jpg

will not work.

 

Save your scene and all modified objects – This probably seems obvious, but you must save the changes to your scene file prior to uploading it for the render to reflect your design. We recommend always appending a version number, and increasing that number with each save, e.g., MySceneV01.lws, MySceneV02.lws, etc.

Upload Your Content

The server name iswww.respower.com, and the user name/password is the same as for the web site.
FileZilla – to upload content, simply drag from the file list on the left into the “source” folder on the right.

Create a ResPower User Name – You must have an account with the ResPower Super/Farm before you can upload any content. Sign up here.

Open your FTP client – Although ResPower is officially FTP-client and platform neutral, we will focus on using FileZilla in this HOWTO. FileZilla is free, fast, and reliable – all Good Things. If you are using another FTP client, the concepts are the same, but the screens will look different. Here is an incomplete list of available FTP clients:

Please note that while using Internet Explorer as an FTP client is possible, it is not recommended. Many users experience unreliable connections and inexplicable behavior when using IE. Because of this, ResPower cannot provide technical assistance under these circumstances. If you have problems connecting with IE, please use one of the fine FTP clients listed above.

 

Connect via FTP – The user name and password are the same as the one you use to log in to the web site. The server name is www.respower.com. The site profile looks like the one shown in the diagram; you can download a FileZilla profile here and import it by selecting File->Import from the site manager menu. To connect, select the ResPower connection and click the “Connect” button.

Upload your content – This is simply a matter of finding your content directory in the “local” file system – the one on your computer – and dragging it over to the “remote” file system, the one at ResPower. Be sure to place all of your content inside the source folder at ResPower. Advanced users will want to create separate folders for each project, e.g., /source/project1, /source/project2, etc. If you do this, be sure to make a note of which subfolder you placed your content in for a particular project – this will be the content directory when you get to the Render Wizard™.

Submit your job for rendering

Render Wizard™ – Log in to the ResPower web site using the Login panel at the top left of the page here. From there, choose the “Render Wizard” link.

Fill in the first screen – Provide a description – the value you use here is irrelevant as far as the farm is concerned. Its purpose is to help you to remember which job is which, rather than forcing you to memorize job numbers as they are assigned by the Super/Farm™. Select your .lws file for the “Scene File” field; the “Content Directory” is the “Project Directory” you uploaded previously.

Mac users note: the Super/Farm depends on your scene file having a .lws extension when it builds its list of scenes

For Frame Range, input the start and end frames, along with any Frame Step you might like to use. For example, if you put 20 under “Start Frame,” 100 under “End Frame,” and 10 under “Frame Step,” the Super/Farm will render every 10th frame from 20 through 100. If you need to render an odd selection of frames, you can leave the Start/End/Step Frame field blank and provide an extended selection on the second screen of the Render Wizard. Finally, select your Payment Method and click the “Update” button.

Fill in the Second Screen – Select the appropriate render engine under “Select Render Engine.” This is extremely important – if you attempt to render a LightWave 8.x scene under LightWave 6.x, it will obviously not work.

Frame Ranges You can edit the list of frames that will be rendered essentially indefinitely. You can add new sequences using the “Add Frame Range” section, add a single frame using the “Add Single Frame” section, and a list of frames using the “Add Frame List” section. Each section has its own “Add” button that you must use for the Render Wizard to recognize the request. If you accidentally add frames that you do not want to render, you can select them in the “Current Frame Ranges” list, then click “Remove Selected” to remove them from the list. The contents of this list indicate which frames the Super/Farm™ will render.

Image Settings – You may select the camera name that you would like to render here. If you do not, then the most recently selected view will be used. If you decide to specify the camera here, please remember that the camera name is exceedingly sensitive to case and spaces – ResPower recommends copying and pasting the camera name directly from the scene rather than typing it in. If you decide to input the Image Height and Image Width, remember that those values are in pixels. They are not required unless you are doing a Split-Frame™ render.

Pixel Aspect Ratio is NOT The Same As Image Aspect Ratio! – A common mistake is inputting the image aspect ratio in the pixel aspect ratio field. These values arenot the same thing! Image Aspect Ratio describes the width of the image relative to its height. Pixel Aspect Ratio describes the width of each pixel relative to its height. For example, a 640×480 image designed for computer screens will have an Image Aspect Ratio of 1.3333, but a Pixel Aspect Ratio of 1.0. If you input 1.333 for the Pixel Aspect ratio, your picture will come out heavily distorted.

Split-Frame Rendering - Basically, this takes a single frame and divides it into a grid. Each cell in the grid, or Bucket, is then sent to a separate computer where it is rendered. This is not supported for animations; use it only to render out stills. At this time, automated re-assembly of the final image is in Beta. In the event it does not work properly, you will need to stitch the frame back together using photo-editing software like Photoshop.

To specify the way in which the scene is split, please set the “Rows” and “Columns” values to the number of rows and columns that the farm should split the render into. You should set the pixel overlap to its maximum value in order to avoid the possibility of any artifacts around the edge of each bucket.

The start/end row and column fields are advanced settings that should normally be left blank. These values will reduce the number of buckets rendered to the rectangle that includes only the rows and columns you specify. For example, if you specify a 5×5 split, and set both the start and end column to 2, then the center column of buckets will be rendered.

ResPower recommends using a 1×25 or 1×30 split when performing LightWave Split-Frame renders. This provides a reasonable trade-off between the overhead of splitting the job up and the acceleration provided by distributing the render. The use of multiple columns in LightWave Split-Frame renders has resulted in polygons failing to appear in the output render; please do not specify more than one column for LightWave renders.

Hang Detection – You can typically ignore this section, unless technical support staff tells you otherwise.

Monitor your job’s progress

Once your job has been submitted, you can monitor its progress and perform special actions using the Jobs page.

To open the jobs page, click the “Jobs” link at the top of any Super/Farm page. Each row contains a job that you have submitted. Jobs include rendering submissions, frames, and certain other commands, notably: unpack, rar, and zip.

The table has several columns:

  • Job #
    This is a unique number identifying your job. These numbers increase sequentially – at the time of this writing, there were 9,438,784 jobs.
  • User
    This should reflect your user name. If you have turned on “Show Other Users”, this column and the “Description” column will be merged for jobs belonging to other users and will contain the phrase “Other User”
  • Description
    This is the description that you provided on the Render Wizard, or one generated during the submission of other types of commands.
  • Status
    This set of columns tells you the current state of your job. If the job is a render, it will be divided further into the columns listed below. Each column will contain a number indicating the number of sub-jobs (frames, buckets, etc) that are currently in that particular state. For example, if you submit a render with 1000 frames, the “waiting” column will have 1000 in it until computers start running those frames. 

    • waiting
      Jobs in this column are waiting for computers to process them.
    • running
      Jobs in this column are being processed by computers.
    • aborted
      Jobs in this column are aborted.
    • dead
      Jobs in this column have failed multiple times, and the Super/Farm has decided not to try any further.
    • complete
      Jobs in this column have finished running.
    • total
      This is the total number of sub-jobs.

    You can drill down to see details on your job by clicking the number of sub-jobs in a particular column. For example, if you have submitted a 1000 frame job, and 5 frames have died, you can click on the “5″ in the “dead” column to get a list of the dead frames.

  • Credits
    This is the running total of Credits used by your job (and its subjobs). The number credits used to render a job is calculated using a formula available on the “Purchase Discounted Credits” page. It is important to note that this number can increase fairly rapidly, and can decrease as well. At the time of this writing, the ResPower Super/Farm is 3.3 THz in size. This means that it can, in theory, go through 3,300 Credits every hour. The actual rate of increase is related to the number and speed of the computers working on your job, and the type of render engine you are using. For example, a 10 frame animation using the most expensive render engines ( e.g., Brazil r/s, finalRender) cannot increase at a rate in excess of 60 Credits every hour, because the Super/Farm currently consists of nodes less than or equal to 6 GHz in speed and the Render Engine Factor for these is 1.00. Similarly, a 10 frame animation using the least expensive render engines (Lightwave or Vue) cannot increase at a rate in excess of 30 Credits every hour because the Render Engine Factor for these is 0.50.
  • Actions
    This column contains a “Go” button and a drop-down list of commands you can perform on a job. Certain commands only show up in certain situations. For example, if your job has no waiting or running sub-jobs, the “abort” command does not appear. 

    • View Log
      Brings you to the job’s log page. If the job has sub-jobs (e.g., frames or buckets), you can get to their logs using the table at the bottom of the log page.
    • What’s Wrong
      Generates a customized report to explain why your frames have died. Many error messages have links explaining how to work around them.
    • Restart Frames [Aborted/Dead]
      This command takes all frames in the Aborted or Dead columns and moves them back to the waiting column.
    • Abort
      This command aborts all sub-jobs, or if the job itself is processing, aborts the job. The farm charges you through the time when you abort.
    • Zip Output
      Causes a job to be submitted to the farm, which generates a series of .zip archives of the output folder. Each archive contains 20 frames, so if your FTP client stalls or you lose your connection, you do not have to start at the beginnging.
    • Rar Output
      Causes a job to be submitted to the farm, which generates a series of .rar archives of the output folder. Each archive contains 20 frames, so if your FTP client stalls or you lose your connection, you do not have to start at the beginning. Furthermore, Rar gives higher compression ratios than .zip.
    • Make AVI
      New! Causes a job to be submitted to the farm, which generates a .avi file from the .bmp’s in the output folder. The .avi is compressed using the XviD codecfor high-quality, deep compression. This option currently only works with .bmps.
      Note: you can download a pre-compiled copy of the XviD codec for Windows and/or MacOS here.
    • Submit Copy
      Submits a copy of the job. Please note that if the job has sub-jobs, they will be copied as well.

 

Pay

Please note: if you are using the Metered Service billing option, you must pay for your job prior to downloading it.

  • Metered Service – On the Jobs page, choose “Pay for Output” from the Actions column, and click Go. Once you have paid for your job, permission to download the results will be granted.

 

Download the frames

Use your FTP client to retrieve the frames. They will be in the /renders/<job-number> folder, and named “frame_<number>.<ext>” You can compress your files prior to downloading using the “Zip Output” or “RAR Output” commands; ResPower recommends using the “RAR Output” option, since it will create a multi-part archive, with each part set to 25 MB.

Known Issues [FPRIME]

• Lack of Progress Information

Versions Affected: [ All ]

FPrime support at ResPower depends on Worley’s WSN module, which doesn’t provide much progress information once a render starts. As a result, there isn’t much progress information for our software to collect and share with you. We’ve sent an email to Worley asking for this feature, but since WSN isn’t officially supported by Worley, we don’t expect an upgrade to contain this any time soon.


Known Issues

•Output folder does not appear on FTP site
•Output folder does not contain all frames on FTP site

There are two possible causes for this:

  • The job does not have any complete frames/buckets. If frames are not complete, there is nothing for the Super/Farm to place into the Output Folder. The Output Folder is only created once at least one frame is complete.
  • Your FTP client is caching the directory listing. Many FTP clients try to save time by storing a local copy, or cache, of each FTP server’s directory listings. Because the ResPower Super/Farm™ is constantly placing files into your FTP directory, this cache can very quickly become out-of-date, which will make your client show you listings that are inaccurate. To solve this, use your client’s “Refresh” option to re-download the directory listing. For FileZilla, the hotkey for Refresh is F5.

  • •Cannot delete or overwrite files on FTP
    •”Permission denied” errors uploading to FTP

    You are not allowed to delete files from your /renders folder; these are automatically cleared out by our automated diskspace recovery tools.

    If you are trying to delete files or folders from your /source folder, or are attempting to upload into a subfolder of your /source folder, there are several likely culprits:

  • The server thinks they aren’t your files. When you unzip files, they show up on the server as belonging to a different user from you. To fix this, use the “Take Access” link available on the web site once you have logged in
  • Your FTP client is not descending into the directory. One of the reasons ResPower recommends FileZilla is that it will descend into directories to delete them.
  • Your FTP client is not deleting hidden files. One of the downsides of FileZilla is that it does not list hidden files by default, and so even if it descends into a directory, it could miss those files, preventing FileZilla from deleting the directory itself. To fix this, we recommend changing two settings. First, choose Edit->Settings from the menu. Then select Connection-> Directory Cache in the tree on the left. Make sure that “Don’t Use Cache” is checked. Select Interface Settings->Remote File List from the tree on the left. Make sure that Always show hidden files is checked. Once you have made these configuration changes, and used the “Take Access” button, you should be able to delete files in your /source/ directory at will.

  • • Render Wizard Doesn’t List Scene File From Macintosh

    Versions Affected: [ All ]

    This is most likely because your scene file has no extension. For example, it might be named “myscene” rather than “myscene.lws”.

    The Macintosh is peculiar in the computing world in that it stores metadata in the file system to determine the type of the file rather than relying on the file’s extension. The Render Wizard™ scans your source/ folder to find files that end in the .lws extension that LightWave uses on other platforms. Please rename your scene file to have a .lws extension and refresh the Render Wizard.


    • Split-Frame Rendering With More Than One Column Yields Missing Textures/Polygons

    Versions Affected: [ Unknown - Probably All ]

    When you use split-frame rendering with a Lightwave scene, you change the “Rows” and “Columns” values in the split-frame rendering section of the Render Wizard. We have seen situations where changing the “Columns” value causes Lightwave to silently fail to apply texture maps to polygons that span the column boundary, or to simply drop the polygons altogether.

    The workaround is simple enough – do not change the Columns value; instead, increase the number of Rows.


    • Quicktime Textures Fail to Render on Super/Farm™

    Please do not expect QuickTime textures to render properly with LightWave on the Super/Farm™. You need to cnvert them to image sequences or AVI’s (we recommend DiVX-encoded AVI’s for tight compression.

    If you have already rendered out a sequence and your QuickTime textures did not show up, you do not have to re-render the entire sequence. You can set all objects other than those with the QuickTime texture to be “Unseen by Camera,” render the sequence using image sequences or AVI’s, and then composite the animated textures in place.


    • FBM HyperTexture breaks on Super/Farm™

    Versions Affected: [ 7.5 ] [ 7.5b ] [ 7.5c ]
    These settings cause Lightwave to randomly leave the FBM texture off of the hypervoxel.(Lightwave 7.5c)
    These are the recommended workaround settings for the ResPower Super/Farm™(Lightwave 7.5c)

    Lightwave appears to have a bug in the interaction between its FBM procedural texture and hypervoxels. The symptoms of this bug are that Lightwave will randomly fail to apply the texture, yielding Hypervoxels with a “flickering” texture. One frame will have the texture applied; the next will not. Re-rendering an affected frame will often fix that frame. When “bad” FBM settings are specified, the failure rate is as high as 50%.

    ResPower has investigated this bug and determined that having a value of “4″ in the frequencies field causes this problem. The default value of “3″ appears to work. In addition, it may be necessary to set the “Texture Amplitude” field above “100%” in order to trigger this bug.

    The workaround is to keep the FBM Frequencies set to 3, the amplitude at or below 100%, and then use the texture scale settings to compensate.


    • Simbiont 2.52 Stores Fully-Qualified File Names

    Versions Affected: [ 7.5c ]

    This means that your darktree shader files cannot be found by the ResPower Super/Farm™ unless you mimic our file system. Please see the Simbiont Support Page for details


    • Baking Plug-in causes dead frames

    The LW_Baker plug-in that comes with Lightwave does not support running in the Lightwave Screamernet module, lwsn.exe. As a result, any scene that references LW_Baker will generate an error message in its output. If our farm detects any error messages, it treats the frame as failed and does not charge your account.

    If you have any LW_Baker plug-in references in your scene, please remove them before uploading to the ResPower Super/Farm™.


    • Particle FX Plug-In Requires .pfx File

    The reason this is necessary is that without the .pfx files in place, some customers were seeing their particles jitter around as different computers did the particle-position calculations at render time.

    To create a .pfx file for use with the ResPower Super/Farm™:

    1. For each FX Emitter reference in the scene,
      1. Pull up the FX_Emitter panel
      2. Go to the “File” tab
      3. Click on “Save Motion”
      4. You should upload your .pfx file to the root of your content directory. [This isn't strictly necessary, but placing them here will save a little bit of render time at the very beginning of your project. It will not affect the speed of each frame - just the amount of time to distribute your render.]

     


    • Macintosh .PCT files “ignored” when used as textures

    .PCT formatted textures generated on a Macintosh are not applied during render. Our only assumption is that this is a bug in Ligthwave dealing with byte ordering. ResPower recommends that you use more widely used formats like JPEG, TGA, or TIFF.


    • Macintosh .PCT files unsupported for RGB/Alpha output

    .PCT formatted output is unsupported on the ResPower Super/Farm™. ResPower recommends that you use more widely used formats, e.g.: JPEG, TGA, or TIFF.


    Layered Procedural Textures Randomly Fail to Apply

    Versions Affected: [ 7.5x ] [ 8.x ]

    Lightwave randomly fails to apply layered procedurals. There is no indication in the screamernet text output that a problem has occurred, and screamernet indicates that everything behaved normally, so this is *not* a problem that ResPower can automatically detect. This is a bug in Lightwave. ResPower has made contact with Newtek to attempt to get this fixed, but has not received a resolution yet.

    ResPower recommends that you bake your procedurals into images in order to avoid experiencing this bug.

    It would appear that this issue has been resolved in Lightwave 8.0. If you haven’t upgraded yet, this alone is worth it! UPDATE: Either it isn’t resolved, or 8.5 has re-introduced the bug.

    Update: We recently found this tidbit on Google. Somebody by the name of “arsad” posted this list of affected procedurals. Part of his message indicated that you should not mix LightWave version numbers. We have not verified this list’s accuracy at all:

    • (LW8 VS LW8.01) STclouds
    • (LW7.5c VS LW8) STclouds, Coriolis, Cyclone, Dented, FBMNoise, HybridMultifractal, Multifractal, Puffyclouds, RidgedMultiFractal, Turbnoise
    • (MAC VS PC & AMD) Coriolis, Cyclone, Dented, FBMNoise, HybridMultifractal, Multifractal, Puffyclouds, RidgedMultiFractal, Turbnoise
    • (LW8 VS LW7.5b) STclouds
    • (LW7.5 VS LW7.5c) HybridMultiFractal

     


    Error reading scene – could not open file

    The most likely cause of this error is that your scene’s file name has characters that are not legal for the Windows-based render nodes that ResPower uses. For example, if you uploaded your scene as “/source/Y:/user/source/scenes/scene.lws” then that file is named “Y:\source\Y:\user\source\scenes\scene.lws”. The second “Y:\” in the file name will confuse Windows.

    The second most likely cause of this error is that your scene’s file name has characters that are not legal for our Linux-based file servers.

    All in all, it is best to stick with non-accented letters, digits, the underscore character, and “.” in your file and folder names.


    Error reading scene – no output files listed
    • Cannot Insert Output Filename Format Yet

    These settings tell Lightwave to save both the RGB and Alpha channels. (Lightwave 7.5c)

    This error occurs because your scene does not have any output file specified. Please specify an output file (still sequences only, not .avi, .mov, or other single-animated-file formats), re-upload your .LWS and try again.

    When you render a Lightwave scene, you must specify an output file or else Lightwave will happily render and throw the results away upon completion. Obviously that is not something you want to have happen when you are paying for the render time, so our farm checks to make sure that you have an output file specified when you render.

    If no output files are specified, our farm will fail to distribute your render to any render nodes, and your entire scene will show up on the “Jobs” page as dead. If you then look at the log for your scene by selecting “View Log” and “Go” on the Jobs page, you will see logs similar to this:

    i 0 2 NODE186 2004-01-12 03:54:05 Processing task (2004.1.7.172)
    i 0 8 NODE186 2004-01-12 03:54:06 Found 1 scene(s) in task to render.
    e 0 2 NODE186 2004-01-12 03:54:06 Task processing failed:
    Error reading scene – no output files listed[0x80020009][Task Processor]
    (std exception:_STL::runtime_error)

    Unless you specify it in Lightwave, the farm has no way of knowing what output file type you would like to have. So, you must be sure to configure some output files. In Layout, select “Rendering”, then “Render Options…” from the menu. Select the “Output Files” and make certain that at least one of “Save RGB” or “Save Alpha” are turned on. Any values specified under “Save Animation” will be ignored, because render farms can only render to a sequence of still images.


    Error: Object loading failed.

    The typical cause of this error is that your .lwo file was corrupted in transit. For example, if you used ASCII FTP mode, this could cause the .lwo to be unreadable on this side. Another possible cause would be that the transfer failed mid-stream and the entire .lwo is not here. Please look at the logs as described above to determine which .lwo was affected.


    • LightWave produces garbage output with Split/Frame.
    • LW_TGA32(.tga) Produces Garbage

    The most common cause of this issue is the type of output file set in the LightWave scene file. Because the farm does some automatic post-process cropping for all Split/Frame jobs, if LightWave produces a file that the cropping software doesn’t recognize, garbage is the result. The recommended workaround is to always output to LW_bmp24 file format. This is a lossless, standardized format that should work in every situation. For customers who require transparency, you will need to enable “save alpha” in LightWave’s render options, and also save it in LW_bmp24 format.

    Comments are closed.