Predix_Logo
  • Categories
    • Questions
    • Predix | Updates
      • Pricing
      • Product
    • Deloitte - Private
    • How-To
    • Accenture-Private
  • Explore
    • Topics
    • Questions
    • Articles
    • Feedback or Feature Requests
  • Sign in
  • Home /
  • Questions /
avatar image
0
Question by Shehrooz Sattar · Jun 22, 2017 at 12:26 AM · time-series-service

Predix timeseries Client Library Issue : java.lang.OutOfMemoryError: unable to create new native thread

Hi, I am trying to query about 22620 tags from a Time-series at one instance. the Micro service is running with 5G memory and 2 instances. i am using following configurations

 predix.timeseries.maxTagsPerQuery=20
 predix.timeseries.maxIngestionMessageSize=511000
 # Predix Timeseries configuration. <<<<< 511 kbytes max
 
 #Purchased Plan restrictions. >>>>>
 # MODIFY TO SUIT YOUR PLAN. GOING OVER MIGHT RESULT IN ADDED CHARGES
 plan.ingestion.concurrent.connections.max=2
 plan.query.concurrent.connections.max=400

So my understanding is the library can establish max 400 connection at any time with the timeseries and can fetch only 20 tags per connection.

Sometimes Micro service gave me the following error

 2017-06-22 06:58:44.726 ERROR 29 --- [io-63784-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed; nested exception is **java.lang.OutOfMemoryError: unable to create new native thread] with root cause**
     at java.lang.Thread.start(Thread.java:714) [na:1.8.0_101]
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_101]
     at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:81) ~[spring-web-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:311) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:349) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:392) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
     at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at java.lang.Thread.start0(Native Method) [na:1.8.0_101]
 **java.lang.OutOfMemoryError: unable to create new native thread**
     at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.14.jar!/:8.5.14]
     at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:677) [tomcat-embed-core-8.5.14.jar!/:8.5.14]


I searched it on google and find out ... This is not a memory problem, but an operating system resource problem. You are running out of native threads.

  1. how many threads the Docker Container will allow your JVM to use?

  2. According to my understanding 400 threads are not too much, right?

  3. Is there any way around this?

Your Help is much appreciated...!

Comment
Add comment · Show 2
10 |1200 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Shehrooz Sattar · Jun 28, 2017 at 11:05 PM 0
Share

@Tom Turner @Siva Balan Can someone help me...?

avatar image Tom Turner ♦♦ Shehrooz Sattar · Jul 04, 2017 at 10:38 AM 0
Share

Hi, I think this might be using the timeseries-client from the docs that doesn't give out the source code?

My team supports the timeseries-bootstrap SDK. Have you tried that one? I know that has been used successfully in many apps. That SDK is outfitted with a connection pool which would be a better way to help with thread management and memory constraints.

In your particular case, if I'm not mistaken, there could be issues with the number of open file handles or other settings that your User in your OS is allowing. In linux, ulimit -a will tell you the current settings. Something I might have an encountered a decade ago comes to mind where a thread limit was affected by this.

1 Reply

  • Sort: 
avatar image
1

Answer by Thuan Truong · Jun 30, 2017 at 03:17 PM

Hi,

How do you obtain the timeseries connection? could you pls post the source code here? (how do you instance the tenantcontext and Client connection)

I'm asking that because I experienced a problem with connection management in timeseries-client lib.

Thanks,

Thuan

Comment
Add comment · Share
10 |1200 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Follow this Question

Answers Answers and Comments

74 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Unable to ingest timeseries data to websocket using predix starter kit 2 Answers

Time series - Operational queries - ordering-storage cost-high frequency store 1 Answer

wind data is not posted to the cloud with error Server error, status code: 400, error code: 210003, message: The host is taken: wind data-service this please help me to find solution 3 Answers

Unable to display the timeseries data on px-chart using iron-ajax tag 1 Answer

Data ingestion to TimerSeries service from NodeJS 2 Answers

GE Monogram
  • Legal
  • Cookies
  • Forum Terms
  • Contact Us
  • Copyright © 2017 General Electric Company. All rights reserved.


Enterprise
Social Q&A

  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Create an article
  • Submit your feedback or feature request
  • Categories
  • Questions
  • Predix | Updates
    • Pricing
    • Product
  • Deloitte - Private
  • How-To
  • Accenture-Private
  • Explore
  • Topics
  • Questions
  • Articles
  • Feedback or Feature Requests