Results 1 to 7 of 7

Thread: OpenCL kernel takes more than 36 hours to compile on Arria 10

  1. #1
    Join Date
    May 2018
    Posts
    6
    Rep Power
    1

    Default OpenCL kernel takes more than 36 hours to compile on Arria 10

    Dear community,

    Before the weekend I started the compilation of my OpenCL kernel using the Intel SDK for OpenCL for the Arria 10 FPGA board. I had compiled an unoptimized version of this kernel before, and that took about 4 hours of compiling and synthesis. When I checked the status of the compilation yesterday (more than 36 hours after starting), I saw it was still busy, and the quartus_sh_compile.log (see attachment) seemed to be stuck in the middle of a line of output. I read on this forum that for the Arria 10 it should take at most around 12 hours to synthesize a large design, so this made me think my compilation was just hanging.

    The compilation used about 28GB of the 64GB available RAM of my build machine, and when it was stuck there was only 1 core being used by the `quartus_syn top -c top_synth` process. If it is of any help, the emulation and verification of the kernel was fine, and gave the same results as the (compilable) unoptimized version of the kernel. The main optimizations are the use of local memory and loop unrolling, and I also changed from a task kernel to ND-Range kernel type. The resource usage is significantly more than the unoptimized version, but still estimated to be within the available resources on the Arria 10 (as you can see in the Optimization Report attached as "my_kernel.txt").

    I don't know what I can do to further debug this as there is no output, so I hope there is someone on this forum that maybe experienced this before, and could help me out.

    Cheers!
    Attached Files Attached Files

  2. #2
    Join Date
    Jan 2017
    Posts
    692
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    Well, place and routing large designs on Arria 10 could take 20 hours or maybe even more, but 36 seems to be very extreme, especially since it seems the process has not even gone past synthesis in your case. This could be transient issue or a bug in the synthesis engine. I cannot think of any specific recommendations other than cancelling and retrying the compilation.

  3. #3
    Join Date
    May 2018
    Posts
    6
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    Thanks for your reply!

    Quote Originally Posted by HRZ View Post
    ... cancelling and retrying the compilation.
    That is what I did, but now, after 24 hours, it got stuck again on the same line in the quartus_sh_compile.log... On another machine I started another compilation, but unrolling a certain loop N times instead of fully. That compiled and synthesized within 4 hours... So apparently it is a resource issue? Which is weird, because the Optimization Report shows the following resource utilization:

    +--------------------------------------------------------------------+
    ; Estimated Resource Usage Summary ;
    +----------------------------------------+---------------------------+
    ; Resource + Usage ;
    +----------------------------------------+---------------------------+
    ; Logic utilization ; 55% ;
    ; ALUTs ; 24% ;
    ; Dedicated logic registers ; 32% ;
    ; Memory blocks ; 82% ;
    ; DSP blocks ; 25% ;
    +----------------------------------------+---------------------------;

    I read somewhere that the estimated resource usage could be off significantly, so maybe in reality it is allocating more than available. But I would expect an error message in that case

  4. #4
    Join Date
    Jan 2017
    Posts
    692
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    Yes, the resource estimation can be significantly off. However, if a resource is overutilized, the process will fail during fitting with a clear error message. The only cases I have seen where placement and routing continues forever is if the design narrowly fits on the FPGA, but the routing never succeeds and keeps retrying to no avail. Can you sort the files generated by the compiler based on time and tell me what is the last file that follows the pattern of top.X.summary (X: map, merge, fit, flow, sta)?

  5. #5
    Join Date
    May 2018
    Posts
    6
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    ls -tr path/to/generated/files | grep top

    Code:
    top.v
    top.sdc
    top_post.sdc
    top_synth.qsf
    top.qsf
    top.qpf
    There don't seem to be any files generated that match the pattern you described.

  6. #6
    Join Date
    Jan 2017
    Posts
    692
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    Well, then it is really getting stuck during synthesis, which is VERY strange and uncommon. You can probably try using one of the newer or older versions of the compiler if you have a compatible BSP and see what happens. You can probably also complain to Altera, but they will likely not pay much attention unless you are using the latest version of the compiler.

  7. #7
    Join Date
    May 2018
    Posts
    6
    Rep Power
    1

    Default Re: OpenCL kernel takes more than 36 hours to compile on Arria 10

    Ok, thanks a lot for your help. I will try to ask the system administrator if he can install the latest version of the SDK. If I get more news on that I will certainly post it here.

Similar Threads

  1. Replies: 0
    Last Post: December 17th, 2017, 03:51 PM
  2. opencl compile error
    By saewon1000 in forum OpenCL
    Replies: 2
    Last Post: November 19th, 2017, 08:44 PM
  3. Replies: 2
    Last Post: October 16th, 2017, 12:14 AM
  4. Replies: 7
    Last Post: November 6th, 2015, 04:52 AM
  5. Same codes on different version of Quartus, one takes forever to compile.
    By ertss in forum Quartus II and EDA Tools Discussion
    Replies: 3
    Last Post: January 20th, 2011, 07:57 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •