distirbution of traces/span  amongst collector

Requirement - what kind of business use case are you trying to solve?
Are collector  load balanced ?

Problem - what in Jaeger blocks you from solving the requirement?

We have our jaegertracing setup working  with back end configured as  elastic search. Currently we have  two collector replica set up .  There are 5-10 services which sends traces to the collector  ( the number of services , keep changing ) .  I see  collectors are not evenly loaded   with traffic.  One collector reaches to the max queue usage  where as  other collector is hardly  using 20-30% capacity .  This causes  the drop from the  collector which is loaded to the capacity .
Can we load balance the traffic  (spans)  amongst the both collector ?  I am not sure if there is any config  and i am missing it.
Proposal - what do you suggest to solve the problem or improve the existing situation?

Any open questions to address

If you are using the Jaeger Agent, you can configure them to use gRPC instead of Thrift (--reporter.type=grpc). You can then either pass a static list of collectors, or use gRPC's notation for discovering the servers (--reporter.grpc.host-port=dns:///service-name:14250).
If your tracers are connecting directly to the collector, only  TChannel is supported at the moment, and it's not possible to load balance individual requests.

we are using agent , will check the configuration . As of now we are managing the  jaeger set up   and there are different team  they are  just bombarding the  traces and spans.  Is there anyway we can do at collector level ?

@prana24 at Uber we recommend all team to use an internal wrapper for Jaeger client libraries, which makes sure that production services are always using remote sampler that pulls sampling strategies from the backend. This way  you can create configuration for the collectors controlling how much each service should sample.
If you have no control over the clients, the brute-force solution is to implement downsampling in the collector (which we do at Uber, but at this point as more of a safety measure). Downsampling is consistently based on trace ID hash, so you don't get partial traces, but downsampling affects all users equally, not just the offending service.
Another approach is throttling clients doing sampling, but it's not currently implemented (#1676).
The best solution imo is tail-based sampling, which Jaeger does not support yet directly, but you can get it with  OpenCensus Service.

We were using  jaeger-agent 1.8.x  ,  i see  grpc  was probably not enabled in that version . I am upgrading agent to latest ( 1.13.x )  .  My collector  is still  1.9.x  , is this version ok  , or i should  upgrade that as well ?

If you can, keep both the collector and the agent at the same version.

Thank you  @jpkrohling   , i have done that  , i have a basic question about dns:///<service_name>:14250  , what is  <service_name>  , here it is the same name  which we get by command  kubectl get service  for collector service ?

It's the DNS name under which the service can be reached. In Kubernetes, this is typically service_name.namespace.svc.cluster.local, but depending on the cluster configuration, you might be able to use only the service name as the hostname, if both the client and the the agent/collector are in the same namespace.
If you are using Kubernetes, I recommend taking a look at the jaeger-operator. Even if you decide not to use it for production, you might benefit from seeing how it deploys Jaeger.

sure, thank you  . I am taking a lookg

Hi ,
I have made changes to my agent .yaml   , somehow it still  sends traffic to one  of the  collector only . it looks like it is not able to dns look up ,   adding my agent.yaml  and agent log here for reference.
019/08/06 11:19:50 maxprocs: Leaving GOMAXPROCS=24: CPU quota undefined {"level":"info","ts":1565090391.0357707,"caller":"flags/service.go:115","msg":"Mounting metrics handler on admin server","route":"/metrics"} {"level":"info","ts":1565090391.0362382,"caller":"flags/admin.go:108","msg":"Mounting health check on admin server","route":"/"} {"level":"info","ts":1565090391.0362995,"caller":"flags/admin.go:114","msg":"Starting admin HTTP server","http-port":14271} {"level":"info","ts":1565090391.0363176,"caller":"flags/admin.go:100","msg":"Admin server started","http-port":14271,"health-status":"unavailable"} {"level":"info","ts":1565090391.0373657,"caller":"grpc/builder.go:75","msg":"Agent requested insecure grpc connection to collector(s)"} {"level":"info","ts":1565090391.041124,"caller":"grpc/clientconn.go:242","msg":"parsed scheme: \"dns\"","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.067472,"caller":"agent/main.go:74","msg":"Starting agent"} {"level":"info","ts":1565090391.0675416,"caller":"healthcheck/handler.go:129","msg":"Health Check state change","status":"ready"} {"level":"info","ts":1565090391.0675752,"caller":"app/agent.go:68","msg":"Starting jaeger-agent HTTP server","http-port":5778} {"level":"info","ts":1565090391.0754118,"caller":"dns/dns_resolver.go:264","msg":"grpc: failed dns SRV record lookup due to lookup _grpclb._tcp.jaeger-collector-dev.sampling.svc.cluster.local on <IP_ADDRESS>:53: no such host.\n","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.119492,"caller":"dns/dns_resolver.go:289","msg":"grpc: failed dns TXT record lookup due to lookup _grpc_config.jaeger-collector-dev.sampling.svc.cluster.local on <IP_ADDRESS>:53: no such host.\n","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.1195319,"caller":"grpc/resolver_conn_wrapper.go:140","msg":"ccResolverWrapper: got new service config: ","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.119683,"caller":"grpc/resolver_conn_wrapper.go:126","msg":"ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:14250 0  <nil>}]","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.1197627,"caller":"base/balancer.go:76","msg":"base.baseBalancer: got new resolver state: {[{<IP_ADDRESS>:14250 0  <nil>}] }","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.1197968,"caller":"base/balancer.go:130","msg":"base.baseBalancer: handle SubConn state change: 0xc00018d560, CONNECTING","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.1241517,"caller":"base/balancer.go:130","msg":"base.baseBalancer: handle SubConn state change: 0xc00018d560, READY","system":"grpc","grpc_log":true} {"level":"info","ts":1565090391.1241896,"caller":"roundrobin/roundrobin.go:50","msg":"roundrobinPicker: newPicker called with readySCs: map[{<IP_ADDRESS>:14250 0  <nil>}:0xc00018d560]","system":"grpc","grpc_log":true}
Also  pasted  here agent.yaml
# Source: jaeger-client-mon/templates/deployment.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata:   name: jaeger-app-1 spec:   selector:     matchLabels:       app.kubernetes.io/name: jaeger-app-1   template:     metadata:       labels:         app.kubernetes.io/name: jaeger-app-1     spec:       containers:       - image: docker.artifactory.prod.adnxs.net/jaeger_client_1         name: jaeger-app-1         ports:         - containerPort: 8080       - image: docker.artifactory.prod.mycompany.net/jaegertracing/jaeger-agent:1.13.1-1-b8a6d4ea680063ab03575e864f233841cfcb45cb58a9c5ddde2e287844c1b679         name: jaeger-agent-1         #args: ["--collector.host-port=jaeger-collector-dev.sampling.svc:14267"]         #args: ["--collector.host-port=pre-prod-open-tracing-collector-dev.adnxs.net:14267"]         args: ["--reporter.grpc.host-port=dns:///jaeger-collector-dev.sampling.svc.cluster.local:14250"]         ports:         - containerPort: 5775           protocol: UDP         - containerPort: 6831           protocol: UDP         - containerPort: 6832           protocol: UDP         - containerPort: 5778           protocol: TCP
Any idea what is wrong here ?

Nothing seems wrong there: gRPC tried to load some extra configuration via DNS but couldn't find anything "extra". As you can see in the following log entries, the connection with the collector was established and is ready:
{"level":"info","ts":1565090391.1197968,"caller":"base/balancer.go:130","msg":"base.baseBalancer: handle SubConn state change: 0xc00018d560, CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1565090391.1241517,"caller":"base/balancer.go:130","msg":"base.baseBalancer: handle SubConn state change: 0xc00018d560, READY","system":"grpc","grpc_log":true}

So, looks like it's working ;-)

It is working but  , i was expecting agent should send traces/span to both the collector  , which currently sending to  only one .   i mean to say it is not load balanced  ,   Am i missing something here ?

You might not see round-robin load balancing, as gRPC will reuse the same pipe for multiple requests, but one easy way to check that it's working as expected is by killing one of the collectors. If the agent switches over to the remaining collector, the load balancing is working.

Oops !!  that  is failover  right ,  that is not loadbalanced ?  I want to avoid  sitaution like this , i have added grafana  images here ,  where  collector1 reaches the max  capacity and   collector2 is sitting idle  , because of this we see  span drops.  ( of course the implementation contains  tchannel communication between agent and collector )  so as advised in this  issue above i am adding  grpc  but somehow still i do not see  spans are being load balanced between  both the collector .

Let me know if  i am doing anything wrong here ?

I just checked the gRPC docs, and it seems that it should indeed be doing round-robin balancing:

It is worth noting that load-balancing within gRPC happens on a per-call basis, not a per-connection basis. In other words, even if all requests come from a single client, we still want them to be load-balanced across all servers.

Source: https://github.com/grpc/grpc/blob/master/doc/load-balancing.md

of course the implementation contains tchannel communication between agent and collector

What do you mean here? The communication between Agent and Collector should be via gRPC, not via TChannel.

@jkandasa, @kevinearls I think one of you ran some tests for this behavior in the past. Can you spot if there's anything missing here?

Just to clear confusion ,  the grafana image which i have posted is  the production problem which i want to solve(  agent , collector  running on 1.8 .x  with tchannel)  .
Since it was recommended that  if we use grpc  with latest version ( 1.13.1 )  we can see traces/span  loadbalanced . I am trying to the same in our dev environment   to see if the traffic is really load balanced.  But  somehow  all the spans are being moved to one collector.
I am just concerned about  how can i get  my traffic loadbalanced  , hence  all the collector  does the work and there is  minimum drops

I'm confused now: you are seeing load balanced traffic in production, but not on your dev environment?

My production version is 1.8  communication with tchannel ,  and the grafana  images  are from production  env.  it shows that  load is unbalanced and also drops.
I want to check if we move to 1.13 .x  with grpc  we can solve the problem in production ,  and that is why i am trying  1.13.1 +grpc  in dev  ( agent.yaml  and log  which i shared )  ,
But in Dev env. also  i do not see   load balanced in traffic ,

@jpkrohling In openshift, we create an additional service(jaegerqe-collector-headless) on the collector for gRPC load-balancing with ClusterIP: None.

That's probably the trick that @prana24 is missing! Thanks @jkandasa!

Vow ..!!  i cant wait  ,  @jkandasa   can you give me more information ?  where do i get it ? any  reference and topology ?

@prana24 AFAIK, there is no specific example to create collector headless service. @objectiser can guide here better.
jaeger-operator creates a headless service by default. Reference in jaeger-operator code
I just copied/modified collector service YAML from generated(by jaeger-operator) service file.
I hope this will work(not tested).
Important line spec.clusterIP: None.
You may add to your existing service and test. If you create a new service named jaeger-collector-headless, do not forget to change it on your agent.
- apiVersion: v1
  kind: Service
  metadata:
    name: jaeger-collector-headless
    labels:
      app: jaeger
      jaeger-infra: collector-service
spec:
  clusterIP: None
  ports:
    - name: jaeger-collector-grpc
      port: 14250
      protocol: TCP
      targetPort: 14250
  selector:
      jaeger-infra: collector-pod
  type: ClusterIP

Thanks @jkandasa  , i will give it a shot today

i have the same problem!!!
##error is
root@ubuntu-165:~/jaeger# kubectl logs productpage-v1-787bcf4b68-j88qj jaeger-agent | grep addrConn.createTransport
{"level":"info","ts":1574242860.4926956,"caller":"grpc/clientconn.go:1191","msg":"grpc: addrConn.createTransport failed to connect to {<IP_ADDRESS>:14250 0  }. Err :connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:14250: connect: connection refused". Reconnecting...","system":"grpc","grpc_log":true}
{"level":"info","ts":1574242861.493872,"caller":"grpc/clientconn.go:1191","msg":"grpc: addrConn.createTransport failed to connect to {<IP_ADDRESS>:14250 0  }. Err :connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:14250: connect: connection refused". Reconnecting...","system":"grpc","grpc_log":true}
##but network was right
root@ubuntu-165:~/jaeger# kubectl exec box2 -- nslookup my-jaeger-collector-headless.kube-system
Server:    <IP_ADDRESS>
Address 1: <IP_ADDRESS> kube-dns.kube-system.svc.cluster.local
Name:      my-jaeger-collector-headless.kube-system
Address 1: <IP_ADDRESS> 10-33-36-204.my-jaeger-collector.kube-system.svc.cluster.local
root@ubuntu-165:~/jaeger# kubectl exec box2 -- nslookup my-jaeger-collector.kube-system.svc.cluster.local

@pujunYang could you please share what's your my-jaeger-collector-headless definition? kubectl get service my-jaeger-collector-headless -o yaml should do the trick. How are you setting it up? Is it via the Operator?

@jpkrohling yes
use Operator start jaeger,
kubectl get service my-jaeger-collector-headless -n kube-system  -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/scrape: "false"
creationTimestamp: "2019-11-20T09:40:13Z"
labels:
app: jaeger
app.kubernetes.io/component: service-collector
app.kubernetes.io/instance: my-jaeger
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: my-jaeger-collector
app.kubernetes.io/part-of: jaeger
name: my-jaeger-collector-headless
namespace: kube-system
ownerReferences:

apiVersion: jaegertracing.io/v1
controller: true
kind: Jaeger
name: my-jaeger
uid: c00e3485-0b79-11ea-ab62-5254006535e0
resourceVersion: "1933"
selfLink: /api/v1/namespaces/kube-system/services/my-jaeger-collector-headless
uid: c0869851-0b79-11ea-ab62-5254006535e0
spec:
clusterIP: None
ports:
name: zipkin
port: 9411
protocol: TCP
targetPort: 9411
name: grpc
port: 14250
protocol: TCP
targetPort: 14250
name: c-tchan-trft
port: 14267
protocol: TCP
targetPort: 14267
name: c-binary-trft
port: 14268
protocol: TCP
targetPort: 14268
selector:
app: jaeger
app.kubernetes.io/component: collector
app.kubernetes.io/instance: my-jaeger
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: my-jaeger-collector
app.kubernetes.io/part-of: jaeger
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}

@jpkrohling  it is was Jaeger.yaml
**apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: my-jaeger
namespace: kube-system
spec:
strategy: production # <1>
allInOne:
image: jaegertracing/all-in-one:latest # <2>
options: # <3>
log-level: debug # <4>
storage:
type: elasticsearch # <5>
options: # <6>
es: # <7>
server-urls: http://elasticsearch-logging:9200
tls:
skip-host-verify: true
ingress:
enabled: false # <8>
agent:
strategy: DaemonSet # <9>
annotations:
scheduler.alpha.kubernetes.io/critical-pod: "" # <10>
**

I have another concerned about load balancing;
We use "--reporter.grpc.host-port=dns:///jaeger-collector-gRPC.service.consul:14250" to get the list of collectors, which is working fine. All collectors receive spans.
The problem;
If we scale out the collectors the agent will never get a new list.
This also means if one or more collectors is removed/offline the list of collectors on the agents will remain the same.
It seems it only resolve the list when the agent starts?
Or am I missing something?

Which version of Jaeger are you using, @parberge? We've bumped the gRPC client version in v1.20.0 which was recently released, and I know it has some improvements in this area, although I'm not 100% sure this case is covered. When fixing #2443, I remember reading that the gRPC client will get a new list of clients only when it runs out of healthy connections, but hopefully this newest gRPC client is smarter.

I'm closing, as I think this has been answered some time ago, but feel free to reopen if there are still questions.

Which version of Jaeger are you using, @parberge? We've bumped the gRPC client version in v1.20.0 which was recently released, and I know it has some improvements in this area, although I'm not 100% sure this case is covered. When fixing #2443, I remember reading that the gRPC client will get a new list of clients only when it runs out of healthy connections, but hopefully this newest gRPC client is smarter.

Not 1.20 that's for sure.
Will test and create an issue if the problem remains. Thanks.

Hey guys - I am seeing some of the same issues. On v.1.19 right now, so will try to do the upgrade. But much like some of the folks are seeing. I am using the jaeger-operator, have the HPA setup for min/max of 2/10, and when CPU gets hammered during our bot/soak tests the collectors scale up as expected, but the agent connections continue to fire spans down their already existing connections. So effectively, feels like more of a fault tolerance setup than a high availability one. @parberge before I go super deep, did you see any positive changes with v1.20.0?

Haven't upgraded yet 😑
Den tors 15 okt. 2020 02:27Josh Kierpiec<EMAIL_ADDRESS>skrev:

Hey guys - I am seeing some of the same issues. On v.1.19 right now, so
will try to do the upgrade. But much like some of the folks are seeing. I
am using the jaeger-operator, have the HPA setup for min/max of 2/10, and
when CPU gets hammered during our bot/soak tests the collectors scale up as
expected, but the agent connections continue to fire spans down their
already existing connections. So effectively, feels like more of a fault
tolerance setup than a high availability one. @parberge
https://github.com/parberge before I go super deep, did you see any
positive changes with v1.20.0?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jaegertracing/jaeger/issues/1678#issuecomment-708756333,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABJQPWI4QULKBMK4ZUWLRWDSKY6WJANCNFSM4IGGIAVA
.

Feel free to reopen this issue you see the same problem happening on v1.20.

Hi folks, unfortunately I see the same behavior as before. Running 1.20.0 container and agent (see below) via jaeger-operator.
Execute our bots firing off spans/traces and can observe the following in our graphs. You can see below we scaled to 4 collector instances, but the agents have no knowledge that they should reconnect and continue to saturate the collectors they are already connected to. The situation makes sense - I'm not missing a configuration in anyway for the collectors to notify the agent they should reconnect when dropping spans?

Containers:
  jaeger-collector:
    Container ID:  docker://98c391498dc8cdb605f5d001350c2451d16800470a7e03c096cab7b808ff7b95
    Image:         jaegertracing/jaeger-collector:1.20.0

...

Containers:
  jaeger-agent-daemonset:
    Container ID:  docker://fa6d25021bc5531053b998789d34fcdb0520d2a8f7907125951c44feeb10ffa4
    Image:         jaegertracing/jaeger-agent:1.20.0

@jpkrohling will try to reopen, need to figure out how =)

I'll check what we can do, but I think the gRPC client might need some time to update the list of backends. In earlier versions, it would update only if all known backends were failing.

OK - I am letting this soak. This may be something unique to how our bots are running also, as they are being spun up asynchronously in a single service, so would make sense that it would overload a single agent. Is the agent designed to have a single connection to a collector at a given point in time? If thats the case, this MAY be OK for us in production when we have bots replaced with real traffic and getting load balanced across our edge service, thus distributing across the agents more naturally.

Is the agent designed to have a single connection to a collector at a given point in time?

I'd have to double-check with the gRPC client load balancer documentation, but I think that's indeed the case. The agent has a list of backends, but will only failover once its "current" backend fails.

@jkandasa do you remember from your load-tests what's the expected behavior here?

Just as a status update, I'm able to reproduce this. Reading some source code from gRPC Go, I was expecting the DNS resolution to happen every 30s, adding the new backends to the list and making them available as subchannels, but looks like it's not happening. I'll check a couple of things, and if they don't work, I'll give the MaxConnectionAge suggestion a try.
The following screenshot shows a situation that started with 10 replicas and later scaled to 20 replicas, expecting the new ones to eventually start receiving traffic.
The disparity of two of the numbers is because I had a wrong configuration. The remaining 8 similar numbers are after adjusting the config to take advantage of both. This is the config used:
receivers:
  otlp:
    protocols:
      grpc:

exporters:
  jaeger:
    tls:
      insecure: true
    endpoint: dns:///simple-prod-collector-headless:14250
    balancer_name: round_robin

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [jaeger]

@jpkrohling,
It's good to hear the status update from you!
FYI, I found a thread that peoples are talking about DNS name re-resolution in grpc repository, and it's very helpful to me to get more understanding about this issue.

I got some time to do some extra experiments, and I agree that setting the MaxConnectionAge would be a good solution. The OpenTelemetry Collector was able to send all the spans to Jaeger Collector, despite the scaling events:
otelcol_receiver_accepted_spans{receiver="otlp",service_instance_id="2582827e-a982-4020-8cb0-fba74c309060",service_version="latest",transport="grpc"} 2.009878e+06

otelcol_exporter_send_failed_requests{service_instance_id="2582827e-a982-4020-8cb0-fba74c309060",service_version="latest"} 4293
otelcol_exporter_send_failed_spans{exporter="jaeger",service_instance_id="2582827e-a982-4020-8cb0-fba74c309060",service_version="latest"} 0
otelcol_exporter_sent_spans{exporter="jaeger",service_instance_id="2582827e-a982-4020-8cb0-fba74c309060",service_version="latest"} 2.009878e+06

With the MaxConnectionAge, here's how the rate of spans per instance looks like:

In the image above, you can see that we had a few nodes at first, ingesting around 20 spans per minute. Then, new nodes appeared, so that each node now takes care of 10 spans per minute.
I'll create a PR adding both MaxConnectionAge and MaxConnectionAgeGrace as CLI options.

Hi @jpkrohling , thanks!
So according to RELEASE.md, this change will be released at 5 January 2022 right?

Correct. If you want to test this change before that, I can tag and generate a container image based on the current main.

Yes, it would be great if I can test this change before that. Thanks!


Supporting floating-point exponents in Operator.power

Summary
Fixes #11454.
Details and comments
In operator.py we simply switch from np.linalg.matrix_power to scipy.linalg.fractional_matrix_power. Thanks to @sadbro for this one-liner. This automatically supports floating-point powers.
In gate.py we switch to using Operator.power method.
Benchmarking
Initially I was sure that the new method (scipy.linalg.fractional_matrix_power, see https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.fractional_matrix_power.html) is faster than the previously implemented method in Gate (based on scipy.linalg.schur), but after running a few more tests (on my windows laptop), I am no longer convinced of this, and this seems to depend on the exponents. I may need to remove the sentence in release notes about the increased performance.
@jakelishman, I know that you have done quite a bit of performance tuning, and so you can probably explain the results I am seeing.
Here is the code that I am running:
def method1(M, t):
    # NEW METHOD
    return scipy.linalg.fractional_matrix_power(M, t)

def method2(M, t):
    # OLD METHOD (adapted from gate.py)
    decomposition, unitary = schur(M, output="complex")
    decomposition_diagonal = decomposition.diagonal()
    decomposition_power = [pow(element, t) for element in decomposition_diagonal]
    unitary_power = unitary @ np.diag(decomposition_power) @ unitary.conj().T
    return unitary_power

time1 = 0
time2 = 0

for seed in range(1000):
	for n in [8, 16, 32, 64]:
		for p in [3, 13, 23, 33, 43]:
			U = random_unitary(n, seed=seed)

			A = U.copy()
			B = U.copy()

			time_start = time.time()
			ret1 = method1(A, p)
			time_end = time.time()
			time1 += time_end - time_start

			time_start = time.time()
			ret2 = method2(B, p)
			time_end = time.time()
			time2 += time_end - time_start

print(f"Method1: time elapsed: {time1:.2f}")
print(f"Method2: time elapsed: {time2:.2f}")

When I run this, I get the time for method1 is 8.36 seconds, while the time for method2 is 25.05 seconds, so we have ~3x performance improvement. However, when I replace the exponents to be floating-point, and explicitly by
[3.1415, 13.5767, 23.4545, 33.4544, 43.122]:

then I get that method 1 takes 119.71 seconds and method 2 26.63 seconds, we we get ~4.5 slowdown.
I still think that whichever method is faster should be implemented in operator.py and gate.py should call that method, but I don't understand which method is better.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.005%) to 87.544%

Totals

Change from base Build<PHONE_NUMBER>:
-0.005%

Covered Lines:
59792

Relevant Lines:
68299

💛  - Coveralls

Thanks, @jakelishman. I have updated the code to use gate's power method for fractional powers (and removed the release note about the speed-up).

Excellent work, thank you! :-)


Does forward.url get written to log

I am unable to get forward.url to work. Does it write to a log or something so that I can see if it is being called? Thanks

No, it doesn't have any logging.

Strange, my forward url doesn't get called. I have just put a link to a page which when called writes a record to the database. If I open the page or refresh it directly in the browser, the record gets added indicating that the page is working fine but when I put the exact same URL in the forward.url and enable forward url, nothing happens. It just doesn't seem to be calling the URL. Any clues?

No ideas. I guess you need to debug the problem. Possibly add some logging.

Logically it can just be a html page that I can call even without any query parameters I guess?

Yes, it can just be a URL without any parameters.

really strange cos I tried calling the plain url with no parameters from different browsers and all work but absolutely nothing happens when I put the same thing in the configuration file. I hope nothing is wrong with these 2 lines from what I can see
    <entry key='forward.enable'>true</entry>
    <entry key='forward.url'>https://trackerdata1.clouddb.com/simplepage.html</entry>

The URL is not accessible, but the format is correct in general.

oh that's because I changed the URL a bit before posting as it is used for other purposes too. My actual code has the working URL. Now I am trying to put that page on a different server and see the result

And also I should just be able to throw in a html page in the opt/traccar/web folder and call that in the forward URL isn't it? Just curious...I know very little about java/jetty etc

Here is an easy way to test it:
    <entry key='forward.enable'>true</entry>
    <entry key='forward.url'>http://localhost:5055/?id={uniqueId}</entry>

Cool so I just enable     <entry key='osmand.port'>5055</entry>
, change my forward URL to http://localhost:5055/?id={uniqueId} and see if it gets logged. Thanks :+1:

Yes, you should be able to see forwarded data in the log as HEX messages.

excellent thanks

Just curious about the semicolons in the forward url
http://example.com:8080/gprmc/Data?id={uniqueId}&code=0x0000&gprmc={gprmc}
And also do you have an example of a real forward url with all the parameters substituted for real values? I am almost there but I still have a bit of difficulty to make forward url work correctly. I am calling a Server side script to transform some the params and post to NoSQL db
Thanks
M.M

I figured that out. Seems to work well now. Thanks anyways :-)

@m-menon Could you please share how u figured out? Im trying to do similar but still having bi of difficulty to make forward work. With example code traccar server doesnt even start. Thanks

It should start, as long as your config file is a valid XML. If something doesn't work, you should check logs for errors.

It doesn't starts when conf/default.xml server config section looks like this

<?xml version='1.0' encoding='UTF-8'?>

<!DOCTYPE properties SYSTEM 'http://java.sun.com/dtd/properties.dtd'>

<properties>

    <!-- SERVER CONFIG -->

    <entry key='web.enable'>true</entry>
    <entry key='web.port'>8082</entry>
    <entry key='web.path'>./web</entry>

    <entry key='geocoder.enable'>true</entry>
    <entry key='geocoder.type'>google</entry>

    <entry key='logger.enable'>true</entry>
    <entry key='logger.level'>all</entry>
    <entry key='logger.file'>./logs/tracker-server.log</entry>

    <entry key='event.forward.enable'>true</entry>
    <entry key='event.forward.url'>http://domain.tld/api/events</entry>
    <entry key='forward.enable'>true</entry>
    <entry key='forward.url'>http://domain.tld/api/positions?id={uniqueId}&deviceId={deviceId}&valid={valid}&fixTime={fixTime}&deviceTime={deviceTime}&protocol={protocol}&name={name}&latitude={latitude}&longitude={longitude}&altitude={altitude}&speed={speed}&course={course}&statusCode={statusCode}&address={address}&gprmc={gprmc}&attributes={attributes}</entry>
    <entry key='event.enable'>true</entry>
    <entry key='event.overspeedHandler'>true</entry>
    <entry key='event.overspeed.notRepeat'>true</entry>
    <entry key='event.motionHandler'>true</entry>
    <entry key='event.geofenceHandler'>true</entry>
    <entry key='event.alertHandler'>true</entry>
    <entry key='event.ignitionHandler'>true</entry>
    <entry key='event.statusHandler'>true</entry>
    <entry key='event.maintenanceHandler'>true</entry>

and logs/wrapper.log20170328 reports:
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at org.rzo.yajsw.app.WrapperJVMMain.main(WrapperJVMMain.java:43)
INFO|21936/0|Service traccar|17-03-28 22:39:47|Caused by: java.util.InvalidPropertiesFormatException: org.xml.sax.SAXParseException; lineNumber: 23; columnNumber: 82; The reference to entity "deviceId" must end with the ';' delimiter.
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at sun.util.xml.PlatformXmlPropertiesProvider.load(PlatformXmlPropertiesProvider.java:80)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at java.util.Properties$XmlSupport.load(Properties.java:1201)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at java.util.Properties.loadFromXML(Properties.java:881)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at org.traccar.Config.load(Config.java:37)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at org.traccar.Context.init(Context.java:186)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at org.traccar.Main.main(Main.java:35)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	... 6 more
INFO|21936/0|Service traccar|17-03-28 22:39:47|Caused by: org.xml.sax.SAXParseException; lineNumber: 23; columnNumber: 82; The reference to entity "deviceId" must end with the ';' delimiter.
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1472)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEntityReference(XMLDocumentFragmentScannerImpl.java:1850)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3061)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at sun.util.xml.PlatformXmlPropertiesProvider.getLoadingDoc(PlatformXmlPropertiesProvider.java:106)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	at sun.util.xml.PlatformXmlPropertiesProvider.load(PlatformXmlPropertiesProvider.java:78)
INFO|21936/0|Service traccar|17-03-28 22:39:47|	... 11 more
INFO|wrapper|Service traccar|17-03-28 22:39:47|waitpid 21936 59136
INFO|wrapper|Service traccar|17-03-28 22:39:47|exit code posix process: 59136 application: 0

Instead, if I change in conf/default.xml file this:
<entry key='forward.url'>http://domain.tld/api/positions?id={uniqueId}&deviceId={deviceId}&valid={valid}&fixTime={fixTime}&deviceTime={deviceTime}&protocol={protocol}&name={name}&latitude={latitude}&longitude={longitude}&altitude={altitude}&speed={speed}&course={course}&statusCode={statusCode}&address={address}&gprmc={gprmc}&attributes={attributes}</entry>

for this: (note: & instead of &)
<entry key='forward.url'>http://domain.tld/api/positions?id={uniqueId}&deviceId={deviceId}&valid={valid}&fixTime={fixTime}&deviceTime={deviceTime}&protocol={protocol}&name={name}&latitude={latitude}&longitude={longitude}&altitude={altitude}&speed={speed}&course={course}&statusCode={statusCode}&address={address}&gprmc={gprmc}&attributes={attributes}</entry>

Now logs/wrapper.log20170328 ends with:
INFO|wrapper|Service traccar|17-03-28 22:54:42|posix_spawn pid 22495
INFO|wrapper|Service traccar|17-03-28 22:54:42|started process 22495
INFO|wrapper|Service traccar|17-03-28 22:54:42|started process with pid 22495
INFO|22495/0|Service traccar|17-03-28 22:54:44|[INFO] StandardFileSystemManager - Using "/tmp/vfs_cache" as temporary files store.
INFO|22495/0|Service traccar|17-03-28 22:54:46|[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
INFO|22495/0|Service traccar|17-03-28 22:54:46|[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
INFO|22495/0|Service traccar|17-03-28 22:54:47|INFO 3/28/17 10:54 PM: liquibase: Clearing database change log checksums
INFO|22495/0|Service traccar|17-03-28 22:54:47|INFO 3/28/17 10:54 PM: liquibase: Successfully acquired change log lock
INFO|22495/0|Service traccar|17-03-28 22:54:47|INFO 3/28/17 10:54 PM: liquibase: Successfully released change log lock
INFO|22495/0|Service traccar|17-03-28 22:54:47|INFO 3/28/17 10:54 PM: liquibase: Successfully acquired change log lock
INFO|22495/0|Service traccar|17-03-28 22:54:50|INFO 3/28/17 10:54 PM: liquibase: Reading from gregorgps.DATABASECHANGELOG
INFO|22495/0|Service traccar|17-03-28 22:54:50|INFO 3/28/17 10:54 PM: liquibase: Reading from gregorgps.DATABASECHANGELOG
INFO|22495/0|Service traccar|17-03-28 22:54:50|INFO 3/28/17 10:54 PM: liquibase: ./schema/changelog-master.xml: changelog-3.8::changelog-3.8-dropuniquetoken::author: Change set changelog-3.8::changelog-3.8-dropuniquetoken::author failed, but failOnError was false.  Error: Can't DROP 'uk_user_token'; check that column/key exists [Failed SQL: ALTER TABLE gregorgps.users DROP KEY uk_user_token]
INFO|22495/0|Service traccar|17-03-28 22:54:50|INFO 3/28/17 10:54 PM: liquibase: Successfully released change log lock
INFO|22495/0|Service traccar|17-03-28 22:54:51|[main] INFO org.eclipse.jetty.util.log - Logging initialized @7863ms
INFO|22495/0|Service traccar|17-03-28 22:54:52|[main] INFO org.eclipse.jetty.server.Server - jetty-9.2.21.v20170120
INFO|22495/0|Service traccar|17-03-28 22:54:54|[main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@67022ea{/api,null,AVAILABLE}
INFO|22495/0|Service traccar|17-03-28 22:54:54|[main] INFO org.eclipse.jetty.server.ServerConnector - Started ServerConnector@43a5b00b{HTTP/1.1}{<IP_ADDRESS>:8082}
INFO|22495/0|Service traccar|17-03-28 22:54:54|[main] INFO org.eclipse.jetty.server.Server - Started @11411ms

However, today's logs/tracker-server.log.20170328 file does not even exists, yesterday's file does.
So far, my API Endpoint (positions) has not registered any activity at all.

I'm actually using postman to test my api endpoint successfully.
Postman uses this:
POST /api/positions HTTP/1.1
Host: domain.tld
Content-Type: application/x-www-form-urlencoded
Cache-Control: no-cache
Postman-Token: dda24dba-bee4-3378-71a5-c6e8c4c0857d

id=3&deviceId=3&valid=true&fixTime=2017-03-27 04:00:06&deviceTime=2017-03-27 04:00:00&protocol=gps103&name=Test Device&latitude=23.68438666666667&longitude=-99.18282833333333&altitude=0&speed=0&course=0&statusCode=0&address=2238 North Drive, Brownsville, TX., USA&gprmc=&attributes={"ip":"<IP_ADDRESS>","distance":0.0,"totalDistance":1.106864443E7}

Today's tracker-server.log file doesn't have a date suffix.

Sorry for the misunderstanding.
Contents of tracker-server.log looks ok. Unless of course I'm not getting it right again.
2017-03-29 01:07:12  INFO: [149102A4] id:<PHONE_NUMBER>66125, time: 2017-03-29 01:07:06, lat: 25.68429, lon: -100.18291, speed: 0.0, course: 0.0
2017-03-29 01:07:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:07:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:08:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:08:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:09:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:09:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:10:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:10:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:11:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:11:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:12:12 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX: 696d65693a3435393731303034303236363132352c747261636b65722c3137303332393039313230392c2c462c3031313230362e3030302c412c323534312e303537352c4e2c31303031302e393734342c572c302e30302c303b
2017-03-29 01:12:18  INFO: [149102A4] id:<PHONE_NUMBER>66125, time: 2017-03-29 01:12:06, lat: 25.68429, lon: -100.18291, speed: 0.0, course: 0.0
2017-03-29 01:12:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:12:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:13:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:13:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:14:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:14:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:15:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:15:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:16:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:16:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:17:11 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX: 696d65693a3435393731303034303236363132352c747261636b65722c3137303332393039313730392c2c462c3031313730352e3030302c412c323534312e303537352c4e2c31303031302e393734342c572c302e30302c303b
2017-03-29 01:17:12  INFO: [149102A4] id:<PHONE_NUMBER>66125, time: 2017-03-29 01:17:05, lat: 25.68429, lon: -100.18291, speed: 0.0, course: 0.0
2017-03-29 01:17:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:17:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:18:59 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:18:59 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:19:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:19:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:20:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:20:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:21:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:21:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:22:11 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX: 696d65693a3435393731303034303236363132352c747261636b65722c3137303332393039323230392c2c462c3031323230352e3030302c412c323534312e303537352c4e2c31303031302e393734342c572c302e30302c303b
2017-03-29 01:22:12  INFO: [149102A4] id:<PHONE_NUMBER>66125, time: 2017-03-29 01:22:05, lat: 25.68429, lon: -100.18291, speed: 0.0, course: 0.0
2017-03-29 01:22:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:22:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:23:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:23:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e
2017-03-29 01:24:58 DEBUG: [149102A4: 5001 < <IP_ADDRESS>] HEX:<PHONE_NUMBER>303034303236363132353b
2017-03-29 01:24:58 DEBUG: [149102A4: 5001 > <IP_ADDRESS>] HEX: 4f4e

What could it be the reason my API Endpoint is not being invoked?

@evallgar Did you try replacing the url with something simpler just to test whether it (forward url) is being called? You could check the logs on either side ( the server where you are forwarding the requests to ) or the traccar logs ( I am not sure though whether traccar logs forward requests in the log but I 'think' it does )
So you could simply take out the existing URL with all those params, and just a simple url with something like http://domain.tld/api/test ( You could create a simple endpoint like this for testing ... that perhaps just logs the requests ) ... and you could then go on adding the params and see where it breaks (if it does).

Thanks @m-menon for your reply. My actual endpoints are for testing purposes. Both are logging whatever happens.
But What I have not tried yet is a more simpler forward url. Will try it right now.
Im also trying to take positions to a mongodb which I understand, you are also onto something like that. How did it go to you?

As these are GET requests (it wasn't POST when I looked at it the last time), you could simply have an endpoint that just returns a HTTP 200 and then try that endpoint using a browser on that very machine  ( where traccar server is installed ) and see if it's accessible and returns an OK status.
Yes I did manage to get the positions to go into MongoDB. Depending on the language you are familiar with and are currently using, lots of REST API frameworks out there.  If you using a hosted MongoDB service like mLab then you could simply use their REST APIs - can be done in a jiffy. But they do mention this though

The second method, documented in this article, is to connect via mLab’s RESTful Data API. Use this method only if you cannot connect using a MongoDB driver

@m-menon

As these are GET requests (it wasn't POST when I looked at it the last time), you could simply have an endpoint that just returns a HTTP 200 and then try that endpoint using a browser on that very machine ( where traccar server is installed ) and see if it's accessible and returns an OK status.

What do you mean?


ci: integration test reports

See https://github.com/posit-dev/posit-sdk-py/runs/25953636649

Creating a new one to deal with Netlify weirdness


[storage-blob] abstract class StorageClient is not exported

The @azure/storage-blob v12 package doesn't currently export the StorageClient abstract class from its entry point.
This is causing a TypeScript compiler issue for one of our samples for the @azure/eventhubs-checkpointstore-blob package because it can't find the protected pipeline member on a ContainerClient, which would be inherited from the StorageClient class.
Sample that causes the TypeScript error:
https://github.com/Azure/azure-sdk-for-js/blob/d66de696b7eb287b89fcd9284d6582077c0e7097/sdk/eventhub/eventhubs-checkpointstore-blob/samples/typescript/src/apiSpecificContainerClient.ts#L26-L48
Details
It looks like the api review tool is warning that StorageClient isn't being exported:
https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/storage/storage-blob/review/storage-blob.api.md

// Warning: (ae-forgotten-export) The symbol "StorageClient" needs to be exported by the entry point index.d.ts

12.1.1 versus 12.1.2
It looks like StorageClient was never exported, however the above sample was working when using storage-blob up to version 12.1.1. It stopped working once upgrading to 12.1.2. The reason it stopped working seems to be that the way the types are distributed changed between these versions.
In 12.1.1, the types for each file were shipped with the package, so even though StorageClient wasn't being exported from the index.ts file, the types for StorageClient were still shipped and the compiler could still find them.
In 12.1.2, the types are bundled up into a single file and exclude anything that's not exported from the index.ts file. This means the types for StorageClient are no longer being shipped.
Fix types by exporting StorageClient
It seems like we should be exporting StorageClient from index.ts. api-extractor has actually been warning that it isn't exported for a while, but due to the way we were shipping types prior to 12.1.2 the compiler was still able to find it. I'm not aware of a reason we would want to hide it.

We chose to not expose it to discourage from attempting to use it because of its good name. One option was to remove it. But then we don't have a place to hold common logic.  Another option was giving it a name that is unlikely be picked from intellisence.

@jeremymeng, @bterlson, @xirzec
What would be the recommendation for someone wanting to customize the pipeline in this case?

Our typical recommendation to customize pipeline is to create a new customized pipeline object and pass into ContainerClient constructor. For example,
https://github.com/azure/azure-sdk-for-js/blob/5897322a3980e1e771b0e37cda7556cdcb5ee66f/sdk/storage/storage-blob/samples/typescript/src/customPipeline.ts#L27
  // Create a default pipeline with newPipeline
  const pipeline = newPipeline(new AnonymousCredential());

  // Inject customized factory into default pipeline
  pipeline.factories.unshift(new RequestIDPolicyFactory("Prefix"));

  const blobServiceClient = new BlobServiceClient(
    `https://${account}.blob.core.windows.net${accountSas}`,
    pipeline
  );

It's by design pipeline property is imutabble for every client object. We don't want it get modified outside once client instance has been initialized.
But ApiSpecificContainerClient is bit different, it tries to modify pipeline in a child class constructor after super call. We don't have officially discussed whether this is a scenario we would recommend. It's an interesting topic.
StorageClient is an abstract base class which holds commen logics for other clients. It doesn't have feature methods. But seems no harm to expose it publicly.

Apologies, it looks like my local project had multiple copies of @azure/storage-blob, after reinstalling packages it looks like StorageClient is actually making it into the bundled definitions file, but pipeline is stripped away because it is marked @internal.
That makes sense actually now that you've mentioned you did not expect pipeline to be exposed like this. It looks like with the old way the types were being bundled, pipeline wasn't stripped out, which is why the sample worked previously.
@ramya-rao-a Maybe for now I should just update the sample for eventhubs-checkpointstore-blob to use the newPipeline method instead of subclassing the ContainerClient. Alternatively we could update the class to also extend an interface that exposes pipeline but that feels riskier given it's meant to be an internal field.

Yeah I'd vote for creating a full customized pipeline in the sample.
The core-http v2 work would make customizing pipelines easier.  Another related feature is multi-version support for storage clients which would deprecate this sample.
It was an Arch Board feedback to remove StorageClient, and we made a trade-off by hiding it.  We could bring the discussion back if there are enough benefits.

Yes, lets use the newPipeline way.

Closing this since the type is exported and I was using a field that was marked @internal.


Odd GPEN Error - 'mask_sharp' referenced before assignment

When running inference.py the following error occurs when Step 5 begins
  File "/inference.py", line 342, in <module>
    main()
  File "/inference.py", line 198, in main
    pred, _, _ = enhancer.process(img, img, face_enhance=True, possion_blending=False)
  File "/third_part/GPEN/gpen_face_enhancer.py", line 116, in process
    mask_sharp = cv2.GaussianBlur(mask_sharp, (0,0), sigmaX=1, sigmaY=1, borderType = cv2.BORDER_DEFAULT)
UnboundLocalError: local variable 'mask_sharp' referenced before assignment

What's likely the cause?

Sorry for the confusing error.
The reason seems to be that one frame in the video fails to detect a face.
The GPEN detection code is here.

Thanks for getting back to me, I bypassed that bit of code but ran into other errors too. Eventually, I got an error telling me "face wasn't detected error", though I wasn't sure that is what was causing the other issues. I'm glad you let me know that's why all of the issues along the way.
In situations like this, what's the best course of action? A different source video?

First you should check if there are any frames in your input video that don't have faces. You should cut out these clips.
In some cases, even though the input video contains full faces, the pre-trained detector still fails to detect them.
In my experience, when the face resolution is too large or the face area proportion is too large, it may not be detected. This may be related to the face detector's training set.

The source definitely has a face in every frame, but I think it may be face area proportion. I cropped the video quite a bit for my test and I'll bet that's why it's having trouble.
Thanks so much for your help and fast response. This project is super exciting and I'm anticipating being blown away by my own results... as soon as I get an actual render. 😅

fix this bug:
mask_sharp = np.zeros((height, width), dtype=np.float32)

fix this bug: mask_sharp = np.zeros((height, width), dtype=np.float32)

fix this bug: mask_sharp = np.zeros((height, width), dtype=np.float32)

Where to write this line of code

@PanJinquan Can you specify where to insert that piece of code?

@Jeriousman

I cant see the temps ?

@Jeriousman

Hi! If i'm using colab version - where should i put this?

@Fillllllllllll
https://github.com/OpenTalker/video-retalking/blob/main/third_part/GPEN/gpen_face_enhancer.py


Evaluation on other domains

Hello I read the whole paper, it is amazing, the math is well written. And on page 21, it says , "Due to the limitation of computation resources, we leave the application to other domains and models to future works." How soon can we see sophia optimizer for other domains such as computer vision , text-to-speech , reinforcement learning or diffusion with the same 2x speed-up from normal adam optimizer ??
Thanks in advance!

Thanks for the interest in our work and this official repo! Sadly we do not have the compute for now to run computer vision, text-to-speech, reinforcement learning or diffusion for now. We will seek collaborations and rely on the community to test Sophia on other domains.


Improve docs & add help on distribution

Codecov Report

Merging #78 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #78   +/-   ##
=======================================
  Coverage   72.49%   72.49%
=======================================
  Files          21       21
  Lines         887      887
=======================================
  Hits          643      643
  Misses        244      244

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cde0a34...77b94cf. Read the comment docs.


Action Items displayed in the UI

Is it possible to get the UI updated so that it can display action items as shown in the demo video?

For sure, we're definitely planning to do this 🙂 if you have any time to work on this, feel free to raise a PR!

+1
How would we feel about allowing action items to be updated via the UI too? (Without going into the admin backend).


Designer View error

I get the error message in the xaml, "Cannot find resource 'thems/materialdesignthemes.popupbox.xaml'.
compile the program and run have no problem, just the designer cannot display correctly.
I tried to fix it, but no luck.
Thanks.

After I deleted all files in bin folder, it works.


Household OCR Excluded Record Types list is missing in NPSP Settings

The list of record types to exclude when auto-creating household member OCR records is missing in NPSP Settings (Donations -> Contact Roles -> Edit).

**lurch: add

Tracking W-025766

I'm seeing this issue too, but only if Customizable Rollups are enabled. The Soft Credit section under Contact Roles in Settings is also completely missing. If you disable the rollups, then the Soft Credit section reappears. In this state, Donor Statistics settings also disappear. Not sure if I should log any of this as a separate issue, or if the same.

Those are actually all valid. Once Customizable Rollups are enabled, some of the rollups related settings in NPSP settings are hidden. This is covered in the documentation. What's missing here are the list of Record Types to exclude when creating OCR records for Household Members. That's not specifically a rollup setting and so it should be visible whether Customizable Rollups is enabled or not.

@julieloyd That is completely normal and expected behavior and I believe it might be in the documentation as well. All of the soft credit rollup values, donor statistics, etc. are incorporated into Customizable Rollups and are therefore not used/hidden. However, the Household OCR Contact Record Type ID setting is not affected or changed by Customizable Rollups and shouldn't have been hidden with the rest. Hope that helps!

Well, I feel sheepish. I broke my own cardinal rule - always check the documentation. Thanks for the quick response!

@force2b Wait. @bethbrains and I think this is actually WAD.
I checked my QA org and I see the same as you (sorry, should have checked first) but that's fine because the org doesn't have any Contact Record Types defined. When I look at an org that does have Contact Record Types, I see this which is correct.

Can you (or @julieloyd) clarify what you're expecting to be different? I thought your screen shot was from pre-CLRP and the entire section was missing, but that doesn't appear to be the case. Thanks.

You're correct @judisohn. That specific list of record types is actually a list of Contact Record Types,  so if there are no Contact Record Types in the org the list will be empty. I'll close this.


chore: add the possibility to disable logging and tracing

This PR proposes to add settings to disable logging and/or tracing when starting a client.

Looks good. Thanks!

Looks good. Thanks!

Thanks for taking a look.


Fix Home page scrolling issues

Notion: https://www.notion.so/paritytechnologies/Docs-Home-horizontal-scrolling-dc5847c3e92742749cfb6dd5a726db66

The only difference in the way this appears from what's currently deployed, is that the browser mockup in the Playground section now fades in from the right starting at the container width, rather than all the way from the right of the view.

@je-boska
it is caused by this AOS transition:

Adding overflow-hidden to the wrapper like shown in the screenshot should be enough to fix it..

@je-boska it is caused by this AOS transition:

Adding overflow-hidden to the wrapper like shown in the screenshot should be enough to fix it..

Yes, I added overflow-x-hidden to the wrapper of that asset.
There were fixed widths on some of the cards in the section above, which was also causing some minor horizontal scrolling. I sorted this by placing them in a grid instead. There is a visible difference in how they appear, but it's very minor.

@je-boska could you test it doesn't break anything when added to to the main container instead?:
<main class="main-container overflow-x-hidden">
Adding only to that AOS wrapper as I suggested crops the transition visually..

I can't find any problems with adding it to the main-container class. Let me know if it's OK to edit the tailwind-preset.css file in this case.
I also fixed vertical scrolling caused by overflow on the section above ("Learn Substrate").

Everything needs to stay in tailwind (or particular used framework in general) context, only if not possible it goes out of the system, add it to the container directly

I can't find any problems with adding it to the main-container class

did you you test it does not break any page on any break point on VA devices?

Tested on mac and iOS, across Chrome/Brave, Safari and Firefox. Couldn't find any breakage.

LGTM


Popular emoji

Coverage remained the same at 62.5% when pulling d6d8d4eba02c363ae70dcb606f4ec09c9f463fcc on andela-sakande:popular-emoji into 2723f854be37121b6addfd8ecafabe49bf0a0a85 on unicodeveloper:master.

LGTM. I'll merge it in now. Thanks @andela-sakande


next release planning

The next release will probably just be a minor release.
Which of the current PRs and issues should we address and try to bring into the next release?
cc @amsul

My opinion is to bring all the pull requests you or @amsul approved. Skimming through them on my phone, i see that you declined many of them.
Once done lets see what issues can be fixed with minimal effort since there are so many of them .
I love your dedication to a jQuery date picker when the creator left it and when the world is leaving jQuery for other frameworks.

@DanielRuf we can do a minor release from what's already in master so the issues that have already been merged are available on npm. Any pending PRs can come in the one after.
It also seems like we have a vulnerable dependency: https://github.com/amsul/pickadate.js/network/dependencies#package-lock.json. It'd be nice to resolve that before the release.

It also seems like we have a vulnerable dependency: amsul/pickadate.js/network/dependencies#package-lock.json. It'd be nice to resolve that before the release.

I do not see the warning, do you mean hoek?
What does npm ls hoek output?


Dockerfiles using deprecated ENV syntax

Per https://docs.docker.com/engine/reference/builder/#env:

The ENV instruction also allows an alternative syntax ENV  , omitting the =. For example:
ENV MY_VAR my-value
This syntax does not allow for multiple environment-variables to be set in a single ENV instruction, and can be confusing. For example, the following sets a single environment variable (ONE) with value "TWO= THREE=world":
ENV ONE TWO= THREE=world
The alternative syntax is supported for backward compatibility, but discouraged for the reasons outlined above, and may be removed in a future release.

Regex used with Visual Studio Code to update the entries:
Find: ^(ENV.*)\s([v0-9.+a-z-]+$)
Replace: $1="$2"


feature/glTF-extractor-for-Maya

Brief description
Allows user to use the extrator to export static or animated glb with Maya2glTF
Description
Allows user to use the extrator to export static or animated glb with Maya2glTF. After enabling the extractor, you can export glb by creating either model, animation or point cache instance.
You need to enable the extractor in Openpype settings

Additional info
You need to install the maya2glTF first so that the extractor can run correctly.
See the link: https://github.com/iimachines/Maya2glTF
Limitation
The UI of maya2glTF would pop up for seconds and close because the system needs to open the UI first to allow run the mel scriptl library from maya2glTF
Testing notes:

install the maya2glTF
enable the extractor in Openpype setting
create instance which is associated to the extractor(model, animation, point cache)
pubish it
you will find the glb file in the folder(s) of the related instance in the publish folder

@moonyuet Nice. I actually have maya2glTF and the Collada exporter to GLTF logic implemented for OpenPype so I can probably merge in some logic. I'm swamped with work these weeks but hope to loop back soon with more details on our implementation.
I didn't need the UI to pop up for it to work, so at the very least that'd be a nice start. I also had support for exporting multiple clips in a single export.


Loading/progress bar

When you execute the script (bin/run), it takes some time to display the result due, depending on the number of repositories. I think it's a good idea to add a progress bar otherwise, it looks like the terminal is stuck.

Displaying something is a good idea. I’m not sure how we’d do a progress bar since we have no idea how long the query is expected to take - also, i wouldn’t want to clear the console after the request comes back, because that’s disruptive, but i also wouldn’t want to leave the progress indicator visible above the table.
Open to ideas!

How about this:
Loading...
Loading completed!

That's fine for initial output, but it'd be nice to find a way to remove that output once the table is ready without console.clear().

@ljharb One of the solution which I suggested to @abdumamdouh yesterday is to save the user's command and show the loading screen.
Once the loading is complete, clear the console. print the user's command and print the data.
npx report-report
Loading...

Once the loading is complete
npx repo-report
{Repositories details}

@khattakdev @ljharb
1-take input, ex: repo-report --version
2-store it in var
3-dispaly "Loading..." while fetching
4-done fetching
5-display "Loading completed!" or skip this step
6-then clear console
7-then display input
8-then display results

It would be ideal to avoid ever clearing the console - i like seeing my previous commands' output, and it's pretty disruptive for a program to clear that.

@ljharb so what do you recommend?

That's what I'm not sure about :-)
I'm wondering if there's something to be done with printf so that we print progress on a single line, and then overwrite just that one line to print the table?


Tests fixing and refactoring

Closes #123, #124, #125, #126, #127.
Simplified tests logic. This PR is needed for the following fixes.

PR-Codex overview
This PR focuses on refactoring and organizing test classes in the codebase.
Detailed summary:

Deleted IntegrationDdrWorkflowBase.ktsrc/test/kotlin/org/objectionary/ddr/integration/IntegrationDdrWorkflowBase.ktsrc/test/kotlin/org/objectionary/ddr/TestNotFoundException.kt.
Added TestNotFoundException.kt class.
Added UnitTestBase.kt interface.
Renamed IntegrationDdrWorkflowTest.kt to DdrWorkflowTest.kt and moved it to workflow package.
Added IntegrationTestBase.kt interface.
Added constructResultPath function to IntegrationTestBase.kt.
Updated imports and implemented UnitTestBase in TempDirectoryBase.kt.
Updated TestBase.kt interface with new properties and functions.
Updated Sources.kt class with new private properties and updated function names.
Updated DdrWorkflowBase.kt with new imports and updated function names.
Updated AttrBase.kt with new imports and updated function names.

The following files were skipped due to too many changes: src/test/kotlin/org/objectionary/ddr/unit/graph/attr/AttrBase.kt, src/test/kotlin/org/objectionary/ddr/unit/graph/inner/InnerBase.kt, src/test/kotlin/org/objectionary/ddr/unit/graph/builder/BuilderBase.kt, src/test/kotlin/org/objectionary/ddr/integration/resolver/ResolverBase.kt

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@Graur please review

@Graur could you review this, please?

@c71n93 What about your unverified commits? Did you sigh your commits by GPG keys?

@c71n93 What about your unverified commits? Did you sigh your commits by GPG keys?

@Graur Yeah, it looks like I had issue with my ssh key. I have fixed it.

@yegor256 could you please take a look?

@rultor mertge

@rultor merge

@yegor256 OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@yegor256 Done! FYI, the full log is here (took me 3min)


Error when using django-allauth in place of django.contrib.auth.

When used in projects using django-allauth, the User relations in several places fai, and yield the following error:
django.core.management.base.SystemCheckError: SystemCheckError: System check identified some issues:

ERRORS:
blog.Article.author: (fields.E301) Field defines a relation with the model 'auth.User', which has been swapped out.
        HINT: Update the relation to point at 'settings.AUTH_USER_MODEL'.
blog.Profile.user: (fields.E301) Field defines a relation with the model 'auth.User', which has been swapped out.
        HINT: Update the relation to point at 'settings.AUTH_USER_MODEL'.

System check identified 2 issues (0 silenced).

Here's the recommended approach to referencing the User model from the Django documentation:
Referencing the User model
I'd really love to see this app in action. It looks amazing, and exactly what I'm looking for. But django-allauth isn't something I can let go of. It's an great collection of integrated user-related functionalities, and would be be an amazing addition to any project wanting to use your blog app. (Like mine. :D )
django-allauth

@changeling
This will require significant changes to the whole project which I will need time to do.

Completely understandable. Do you accept PRs?

Yes I do


Support controlling rendered images and pdf transparent replacement color

This is a proposal for #586, managing all formats supporting transparency.
It allows global configuration as well as local, and tests are implemented to validate the behavior is correct for all supported formats.
(It also fixes a small culture issue in a test that arose on my machine.)

LGTM. i will deploy now


iOS touch events do not properly remove focus rectangle

In iOS, if isNavigatingWithKeyboard is set to true via attached keyboard or a focus event outside of the _isMouseUsedTimer, then _onMouseDown will not set isNavigatingWithKeyboard back to false because it will get caught in the following return exception:
https://github.com/microsoft/keyborg/blob/main/src/Keyborg.ts#L164
Turns out that Android and Windows touch events mimic mouse events and pass a buttons of 1, but iOS touch events have a buttons value of 0, which causes the touch to be treated like a screen reader event.

Just as an addendum here, we also need to make sure iOS + bluetooth keyboard maintains focus outlines, since I think that's the line enabling that behavior now.

My suggestion for how to proceed, based on BizChat wanting this fixed quickly as a ship-blocker would be:

remove the buttons: 0 heuristic entirely for now. Maybe also the clientX/clientY/etc heuristic too, since at a quick glance that didn't seem to be catching any modern SRs.
Look into adding :focus-visible as a focus selector in addition to the tabster data attribute (i.e. so the focus style shows if either the data attribute is present, or :focus-visible matches) in @fluentui/react-tabster

The second one would help us have more robust focus styles in general, and get us out of the trap of trying to use heuristics to detect ATs. That way the data attribute is primarily a fallback for browsers that don't support focus-visible yet

fixed #80


Speed Up Packaging For All Platforms + Have Installers For Mac and Windows

Issue Label: [Enhancement]
Description of bug or feature enhancement:
It takes to long currently to build out a new release even after the code is done (about 2 hours) and it is only because i have to be running the target OS.
Solution lies here https://github.com/electron-userland/electron-builder and in my package.json
OS and OS version:
All Platforms & Architectures Possible
Version of Insomnia Application:
Who knows when this will happen

Mac installer added with https://github.com/sindresorhus/create-dmg

windows installers suck, i think exe for windows will just have to stay


Server error at amnesia

If a user tries to get a reset password email by entering their email address via the amnesia view, if the email address belongs to multiple users in the system (as far as I looked it is not a unique field), an exception is thrown resulting in a server error. We should perhaps make the email field unique and/or catch an exception and tell the user to contact someone for assistance. Example exception:
Internal Server Error: /accounts/amnesia/

MultipleObjectsReturned at /accounts/amnesia/
get() returned more than one User -- it returned 2!

Do we also want to make username a unique field along with email ?

Oops, my initially suggested solution doesn't make sense. We shouldn't be looking at email being unique in invite.models.InviteItem. The invite.forms module already does checking for email and username uniqueness against the existing users in django auth. Instead we could do a filter instead of a get and use the first item in the queryset (usually there should only be one) unless we created multiple accounts for someone directly making use of django admin. Does that make sense?


It doesn't work on the first run

The plugin doesn't work on the first run or right after a hexo clean.
How to reproduce:

hexo clean
hexo generate -> In the public folder the HTML is still there with the ?__inline=true
hexo generate -> Now the  tags are replaced by its inlined equivalent

it's by design, first round generate the base result, second round do the inline job

I think you should put this into the README.md


Can withStyles pass props to styles object?

Currently, I developing a component that requires both props and themes object.
At first, it works great with theme object
const styles = theme => ({
  title: {
    ...theme.typography.headline,
    textAlign: 'center',
    padding: '8px 16px',
    margin: 0,
    color: theme.palette.common.white,
    backgroundColor: theme.palette.primary[500],
  },

withStyles(styles, { withTheme: true })(Component);
....

But I also need access to props in styles object.
I tried example but it not working.
{
 ....
display: (props) => props.display
}

I end up combine react-jss and withTheme to do that

import { withTheme } from 'material-ui/styles';
import injectSheet from 'react-jss';

function withStyles(styles, Component) {
  return withTheme()(injectSheet(styles)(Component));
}

export default withStyles;

....

const styles = {
  title: {
    display: (props) => props.display,
    textAlign: 'center',
    padding: '8px 16px',
    margin: 0,
    color: ({ theme }) => theme.palette.common.white,
    backgroundColor: ({ theme }) => theme.palette.primary[500],
  },

It works but I really miss
  title: {
    ...theme.typography.headline,

We should probably be able to address the issue by making Material-UI using the same context key as react-jss: https://github.com/cssinjs/theming/blob/master/src/channel.js#L1.
Also, have a look at #7633

I have a PR ready with a react-jss interoperability example. I will add that into the docs. cc @kof

Thanks a lot!

@oliviertassinari does the resolution of this mean that it should be possible now to get access to the props within the styles definition? It's not clear to me how...

@pelotom no, withStyles doesn't have access to the properties. But given how much people are asking for this feature. It's something I can prioritize, after the bug fixes. You can use the injectSheet HOC, but it's open the door to multiple  issues: memory leak, hot reloading broken, no classes composition, no internal ref access, broken theme nesting handling. At least, it's some of the issues I have been facing in the past and motivated my rewrite. I think that step by step those issues will be addressed.

@oliviertassinari It's very reassuring to hear that you'll consider prioritizing this! This would make it much easier to customize components. For example, I'd like to have a checkbox with a configurable size (i.e. width & height in pixels):
<CustomCheckbox size={16} />

If we could access props in styles, this would be very simple:
const styles = {
  root: {
    width: props => props.size,
    height: props => props.size
  }
}

or
const styles = props => ({
  root: {
    width: props.size,
    height: props.size
  }
})

and then:
const CustomCheckbox = ({size, classes}) => <Checkbox className={classes.root} />;

export default withStyles(styles)(CustomCheckbox);

For now, do you have any recommendations about how we should approach these types of use cases? Or do you have any estimates of when you might be able to add support for accessing props when using withStyles?

@nmchaves You use case seems to fall perfectly for the inline-style approach, you can find a bit about it in the documentation. FAQ
https://github.com/callemall/material-ui/blob/75a30061e76eae93c711ec202a2c7e4238a4f19a/docs/src/pages/style/SvgIcons.js#L38-L44

Thanks @oliviertassinari ! I was hoping I could accomplish this using withStyles, but inline-styles will work great. And the fact that you're recommending it here + in the docs makes me feel very confident with this decision. Thanks again!

it would be nice to be able to pass a prop (image src) to the style for a backgroundImage

I'd just wrap withStyle
const withStylesProps = styles =>
  Component =>
    props => {
      console.log(props);
      const Comp = withStyles(styles(props))(Component);
      // return
lol
;
      return <Comp {...props} />;
    };

demo: https://codesandbox.io/s/k2y01rj3w7

@caub It's working, but you need to be cautious with this pattern. The injected CSS will grow with the number of instance of the component. It's a duplicate of #7633. I haven't dug into the topic. But I believe @kof version use some performance optimization.

@caub Thanks for sharing!

@oliviertassinari there's this https://github.com/cssinjs/react-jss/blob/master/readme.md#dynamic-values in react-jss, I wonder if if can't be used in material-ui? Also I understand your point where you say inline style prop is perfect for dynamic values, but it's nicer to have all styles definitions in the same places. There's also https://github.com/airbnb/react-with-styles that would handle className and style

I am facing the same issue can some one help me out
`import React from 'react';
import PropTypes from 'prop-types';
import { withStyles } from 'material-ui/styles';
import Drawer from 'material-ui/Drawer';
import AppBar from 'material-ui/AppBar';
import Toolbar from 'material-ui/Toolbar';
import List from 'material-ui/List';
import Typography from 'material-ui/Typography';
import IconButton from 'material-ui/IconButton';
import Hidden from 'material-ui/Hidden';
import Divider from 'material-ui/Divider';
import MenuIcon from 'material-ui-icons/Menu';
import { mailFolderListItems, otherMailFolderListItems } from './tileData';
const drawerWidth = 240;
const styles = theme => ({
root: {
width: '100%',
height: 430,
marginTop: theme.spacing.unit * 3,
zIndex: 1,
overflow: 'hidden',
},
appFrame: {
position: 'relative',
display: 'flex',
width: '100%',
height: '100%',
},
appBar: {
position: 'absolute',
marginLeft: drawerWidth,
[theme.breakpoints.up('md')]: {
width: calc(100% - ${drawerWidth}px),
},
},
navIconHide: {
[theme.breakpoints.up('md')]: {
display: 'none',
},
},
drawerHeader: theme.mixins.toolbar,
drawerPaper: {
width: 250,
[theme.breakpoints.up('md')]: {
width: drawerWidth,
position: 'relative',
height: '100%',
},
},
content: {
backgroundColor: theme.palette.background.default,
width: '100%',
padding: theme.spacing.unit * 3,
height: 'calc(100% - 56px)',
marginTop: 56,
[theme.breakpoints.up('sm')]: {
height: 'calc(100% - 64px)',
marginTop: 64,
},
},
});
export class ResponsiveDrawer extends React.Component {
state = {
mobileOpen: false,
};
handleDrawerToggle = () => {
this.setState({ mobileOpen: !this.state.mobileOpen });
};
render() {
const { classes, theme } = this.props;
const drawer = (

    <Divider />
    <List>{mailFolderListItems}</List>
    <Divider />
    <List>{otherMailFolderListItems}</List>

);

return (

      <AppBar className={classes.appBar}>
        <Toolbar>
          <IconButton
            color="inherit"
            aria-label="open drawer"
            onClick={this.handleDrawerToggle}
            className={classes.navIconHide}
          >
            <MenuIcon />
          </IconButton>
          <Typography variant="title" color="inherit" noWrap>
            Responsive drawer
          </Typography>
        </Toolbar>
      </AppBar>
      <Hidden mdUp>
        <Drawer
          variant="temporary"
          anchor={theme.direction === 'rtl' ? 'right' : 'left'}
          open={this.state.mobileOpen}
          classes={{
            paper: classes.drawerPaper,
          }}
          onClose={this.handleDrawerToggle}
          ModalProps={{
            keepMounted: true, // Better open performance on mobile.
          }}
        >
          {drawer}
        </Drawer>
      </Hidden>
      <Hidden smDown implementation="css">
        <Drawer
          variant="permanent"
          open
          classes={{
            paper: classes.drawerPaper,
          }}
        >
          {drawer}
        </Drawer>
      </Hidden>

        <Typography noWrap>{'You think water moves fast? You should see ice.'}</Typography>

);

}
}
ResponsiveDrawer.propTypes = {
classes: PropTypes.object.isRequired,
theme: PropTypes.object.isRequired,
};
export default withStyles(styles)(ResponsiveDrawer);
`

The injected CSS will grow with the number of instances of the component.

@oliviertassinari injected CSS will grow +- same way html would grow with inline styles. Static styles are rendered in separate sheets and reused across all component instances.

I did like this, though it will be re-render the withStyle from render to render.
import React from 'react';
import {
  withStyles,
  Grid,
  CircularProgress
} from 'material-ui';

const PreloadComponent = props => {
  const { classes,size } = props;
  return (
    <Grid className={classes.container} container justify={'center'} alignItems={'center'}>
      <CircularProgress size={size}/>
    </Grid>
  )
};

const StyleWithThemeProps = (props) => {
  return withStyles(theme => ({
    container: {
      paddingTop: props.size*2 || 50,
      paddingBottom: props.size*2 || 50,
    }
  }),{withTheme: true})(PreloadComponent)
};

const Preload = props => {
  const { size } = props;
  const WithStylesPreloadComponent = StyleWithThemeProps(props);
  return (
    <WithStylesPreloadComponent {...props}/>
  )
};

Preload.defaultProps = {
  size: 20
};

export default Preload;

We can use full pure component to avoid updating
const PreloadComponent = props => {
  const { classes,size } = props;
  return (
    <Grid className={classes.container} container justify={'center'} alignItems={'center'}>
      <CircularProgress size={size}/>
    </Grid>
  )
};

const StyleWithThemeProps = (props) => {
  return withStyles(theme => ({
    container: {
      paddingTop: props.size*2 || 50,
      paddingBottom: props.size*2 || 50,
    }
  }),{withTheme: true})(PreloadComponent)
};

class PreloadFull extends React.PureComponent {

  constructor(props,context) {
    super(props);
  }

  componentWillMount() {
    this.StyledPreloadFull = StyleWithThemeProps(this.props);
  }

  componentWillUpdate(nextProps) {
    this.StyledPreloadFull = StyleWithThemeProps(nextProps);
  }

  render() {
    const { StyledPreloadFull,props } = this;
    return (


how to read django saved data in redis from golang

my django project put data(maybe set or dict) in redis, then I want to read out in golang.
could y pls give me a detailed example.

from the README:
import "github.com/nlpodyssey/gopickle/pickle"

// ...

// from file
foo, err := pickle.Load("foo.p")

// from string
stringDump := "I42\n."
bar, err := pickle.Loads(stringDump)

// ...

so, just read the data from Redis into a []byte, string or os.File and pickle.Load it (or pickle.Loads it).

(closing. feel free to reopen if that wasn't too helpful)


Implement endorse action (#371)

NOTE: These screenshots were taken from separate games, so the game state was a bit different.

See comment about cities should be endorsable (and dogmable)

Yup! I already made that change about an hour ago. I attached a screenshot of the dogma, but the endorsement should work fine too.

do city cards always warn if someone tries to dogma them? (like a demand when you don't meet the criteria)

Ah I forgot to do that. Just pushed a commit to add this warning, thanks!


why readSingleDataBlock and readDataList are private methods

Hi, I am a data engineer in a vehicle company. We use "asammdf" in python to analyze MDF file. I noticed this GO version and try to optimize the processing speed with GO. I found the signal related methods are all private. asammdf in python opened the signal reading methods. Will you consider make it public methods in the future?

Hi @sccds
Thank you for your contribution :)
The methods readSingleDataBlock and readDataList are private because there is no need for the user/you to manage reading either a datalist block (DLBLOCK) or a datablock (DTBLOCK). The function GetChannelSample() already does it for you. When you call GetChannelFunction it will decide for you and already apply the conversion from CCBLOCK
Is there a specific reason for you to directly use readSingleDataBlock and readDataList ?
I can look to put them as Public if you really need it, no worries. Just let me know.

hi，GetChannelSample() is Ok for the known signal name. I am exploring a method to traverse all the signals in the MDF file and store it into OLAP databases (such as Clickhouse).

does the example below fit your needs ?
it's not the most performatic or ideal, but it can help you while I finalize the implementation of a structure that stores signals in a more robust and intelligent way.
// Get channel all samples
channelMap := m.ChannelNames()
for index, channels := range channelMap {
      for _, channel := range channels {
	      samples, err := m.GetChannelSample(index, channel)
	      if err != nil {
		      panic(err)
	      }
	      // put your function to save sample to store desired
              exampleFunctionSaveToOLAP(&samples)
      }
}

Hi I tried this,it returned panic: package not ready to read this file
It looks like the error happened during reading the file:

Yes, the package is not yet ready to read all types of data blocks. I would guess that your MDF file has a zippedblock (not yet implemented). Could you send me a sample file (it can have random values/no conformance values) so I can debug it ? I can take a look and implement the read function by Monday, It will also be useful to use this file in the unit test to help this package, if you authorize it. If that's not possible, send me an e-mail to<EMAIL_ADDRESS>and I'll investigate the file to implement the function you need and then delete the file.
Translated with DeepL.com (free version)

Yes, the package is not yet ready to read all types of data blocks. I would guess that your MDF file has a zipblock (not yet implemented). Could you send me a sample file (it can have random values/no conformance values) so I can debug it ? I can take a look and implement the read function by Monday, It will also be useful to use this file in the unit test to help this package, if you authorize it. If that's not possible, send me an e-mail to<EMAIL_ADDRESS>and I'll investigate the file to implement the function you need and then delete the file.

add fmt.Println(blockHeader) before the switch case and send me the result

Take a look on development branch. I release a new structure much more optimized that runs 10x faster than current version. Now I just need the output from fmt.Println(blockHeader) or a sample file to release the datablock from need by you.
You can take a look on the README file to see the new approach on the code.
you can use like this:
import (
	"fmt"
	"os"

	mf4 "github.com/LincolnG4/GoMDF"
)
func main() {
	file, err := os.Open("sample1.mf4")
	if err != nil {
		panic(err)
	}

	m, err := mf4.ReadFile(file)
	if err != nil {
		panic(err)
	}

file, err := os.Open("sample1.mf4")
	if err != nil {
		panic(err)
	}

	m, err := mf4.ReadFile(file)
	if err != nil {
		panic(err)
	}

// Get channel samples
	channels := m.ListAllChannels()

	for _, channel := range channels {
		samples, err := channel.Sample()
		if err != nil {
			panic(err)
		}

		fmt.Println(samples[:10])
	}
	}

Any update ? I release a new version v1.0.0 and it has a new type of block reading SDBlocks. Also,  I add a print to the block type when you receive the panic: package not ready to read this file, so you should receive the block header output that looks like "##DZ" or "## ± two letters". Let me know if you need help

I suspicious that you are reading a Unsorted File, since the package is not prepared for it yet, you will receive a lot of wrong values from some channels and errors from others. I'm current work on Unsorted files.

Hi, @sccds
I commited a new version in DEV to read UnsortedFiles. I will improve it yet but it can be use. Could you test it to solve your issue ?
Otherwise I will close this issue due inactivity


Clears production of any files, no question asked

I have (had) a habit to save the files regarding production runs into a folder called ehh... production. Containing subdirs per hardware version/revision that was/is to be produced. This includes: schematics, gerber- and position files, parts quotes, 3D/CNC files etc.
By clicking the 'Fabrication Toolkit' icon in my PCB editor, the plugin is so 'kind' to throw all files contained in this directory away for me, with no questions asked. I get the fact that this is open source software without any guarantee and provided as-is.... but still this behavior seems somewhat 'rude' to me. Can the plugin be made to ask for a destination folder or at least ask for confirmation before deleting a whole bunch of files...? Many 🙏🏻

Sorry for that,
that already fixed in the next release (#67), I will publish it this week

Fixed in v2.4.0


Typo in documentation and confusing docstring

This line contains H(s)=\sum_i b[i] s^i / \sum_j a[j] s^i (here is the doc). The power of the second s is inconsistent and I guess it should be a j and not an i.

Moreover, the docstring state that :

where a are elements of the numerator num and b are the elements of the denominator den.

IMO this is very confusing because scipy uses already a as the denominator and b as the numerator in a lot of functions, for exemple in scipy.signal.butter() (here is the doc).
Therefore my proposal is to stick with a as the denominator and b as the numerator.

Maybe I misunderstood otherwise I would be glad to make a Pull Request.

yes, the text isn't even consistent with itself because b is on the top of the fraction.

oh it's already been fixed.   see https://github.com/scipy/scipy/blob/master/scipy/signal/ltisys.py#L581 and http://scipy.github.io/devdocs/generated/scipy.signal.TransferFunction.html

You are right, I was looking at the docs and code from v0.16, my bad.
So nothing to do here, I close.


Validation issue with +1 268 73X XXXX from Antigua and Barbuda (AG)

+1 268 7XX XXXX are mobile phone numbers in Antigua and Barbuda
+1 268 72X XXXX numbers are accepted by this library.
+1 268 73X XXXX numbers are invalid
Reproducible:
https://libphonenumber.appspot.com/phonenumberparser?number=%2B1+268+736+1000
see the contacts of this company:
https://www.digicelgroup.com/ag/en/mobile/contact-us.html
Extra info

http://www.howtocallabroad.com/results.php?callfrom=united_states&callto=antigua_barbuda
http://www.searchyellowdirectory.com/reverse-phone/1268/

Thank you for taking the time to improve libphonenumber!
Could you please provide evidence which says either complete 73 range is valid or specific subranges of 73 (like 731,732..). See our guidelines.
Generally Antigua and Barbuda releases its mobile ranges defining first 3 digits. Earlier official announcement: http://www.itu.int/dms_pub/itu-t/oth/02/02/T02020000080001MSWE.doc. As per existing evidences we can add only 736 sub range.
Please note that you have submitted this information with the knowledge that we
may use, modify, reproduce, publicly display, and distribute the information in
accordance with our open source policies. Please let us know if you did not
intend to submit the information for such use.

Sadly I could not found any newer official document about the area codes. The referenced document from 2003 is obviously deprecated.
I found a Yellow Pages-like site with phone numbers:
http://www.antiguanice.com/v2/directory.php
There are phone numbers listed with 732, 734, and 736.
I also found these area codes (not listed in the document): 465, 720, 776, 783, 785, 788.

Fixed in release 7.6.1
https://libphonenumber.appspot.com/phonenumberparser?number=%2B1+268+736+0000

I think it is only a partial fix.
What about these numbers?

There are phone numbers listed with 732, 734, and 736.
I also found these area codes (not listed in the document): 465, 720, 776, 783, 785, 788.

@pbakondy
True, it is only partial fix. We missed your latest additions.
We will be supporting 732 and 734 ranges also in an upcoming release based on numbers provided in http://www.antiguanice.com/v2/directory.php.
Please note that you have submitted this information with the knowledge that we
may use, modify, reproduce, publicly display, and distribute the information in
accordance with our open source policies. Please let us know if you did not
intend to submit the information for such use.

Fixed in 7.7.1: https://libphonenumber.appspot.com/phonenumberparser?number=%2B1+(268)+732-0059


Detail new guidelines for css refactoring and css variables

Come up with naming conventions and rules for the new css variables that will be added in the style refactoring. Also come up with general practices for the new styles after the refactoring.

Draft 1:
• Try to flatten the structure as much as possible (use css-grid) - accessibility?
• No need to use BEM, simple class name will suffice because of styling encapsulation provided by the shadow DOM (emulated or not)
• Certain features can be defined in the theme and activated via a flag. These flags could even be grouped in a mixin that activates a feature
• Avoid unnecessary markup, avoid outer div. Instead, use the :host selector to apply style to the component
• Try to avoid masking big duplicated chunks of html using css and rather use ts?

Css variables (custom properties)
• A css variable should be provided where a rule is defined and likely to be changed.
• Variables should be formatted like "--cx-{name of the rule}".
• When assigning a value to a rule and also providing a css variable for the same rule, provide the default using the fallback. i.e. `margin: var(--cx-margin, 0 10px 0 10px);`.
• Not everything need a variable. They should be added only if necessary and some styling should remain opinionated and not provide easy configuration.
• We should not add variables without a fallback unless the value is specified at the root of the component:

Bad
.test {
	Margin: var(--cx-color);
}

Ok
:root {
	--cx-color: red;
}

.test {
	Margin: var(--cx-color);
}

Most css variables will be scoped to a class or element but we should still aim to provide variables with a global scope. To distinguish the two types of variables, we should use a separate naming convention for global variables. These variables should be named --cx-theme-{name of the rule}.
Bonus
Suggestion: Use css variables in the theme. At the moment, we are mostly using sass variables with at static value. These values should be replaced with a css var and a fallback. This allows easier customisation for theme variables used inside of the styles lib rather than outside. For example, the font sizes are used inside the type mixin. Therefore there is no easy way to change the value of those variables so they have an effect inside the mixin. To fix this, we can do : $h1-font-size: var(--cx-theme-h1, $font-size-base * 2.25);. Afterwards, this variable can be modified in the :root of the app styles.

@LTiger14 I actually thought this ticket was going to be only about the list of css variables :) That being said, this list is also needed, and should be based on https://github.com/SAP/cloud-commerce-spartacus-storefront/issues/11#issuecomment-438174669.
we could either change this ticket, or start with a new one to come up with the css variable list. Where I think we should have 2 groups:
Theme variables:

--cx-theme-border-radius
--cx-theme-transition-time
...

Component variables:

--cx-width
--cx-margin
...

The effort of this/that ticket would be to come up with an initial list.
btw. I wouldn't mind to find a shortcut for --cx-theme...

I'd prefer to transition this ticket and have it more focused on getting the list.

Like @tobi-or-not-tobi mentioned, we should have to groups of css variables.
Theme variables:

--cx-theme-primary, --cx-theme-secondary (colors)
--cx-theme-border-radius
--cx-theme-transition-duration
--cx-theme-animation-duration
--cx-theme-gutter-width
--cx-theme-font-weight-normal/bold...
--cx-theme-spinner-size ?

Side note: In the styles library theme, there are multiple variables that aren't !default and therefore can't be easily overridden in their use within the styles lib. If the objective is also to override them we could add the following:

--cx-theme-font-size-base
--cx-theme-h1-font-size (h2,h3,h4...)
--cx-line-height-base
--cx-heading-line-height

Similarly :

--cx-theme-input-heigh (for example)
i.e. $input-height: var (--cx-theme-input-heigh, 48px);

Component variables:

--cx-margin
--cx-padding
--cx-height
--cx-width
--cx-min-height
--cx-min-width
--cx-border-width/style/color
--cx-grid-row/column
--cx-grid-template-columns/rows
--cx-justify-self/--cx-align-self/--cx-place-self
--cx-justify-items/--cx-align-items
--cx-text-align
--cx-color
--cx-background-color
--cx-grid-column-gap/--cx-grid-row-gap
--cx-flex-direction
--cx-justify-content
--cx-align-self/items/content
--cx-zoom
--cx-opacity
--cx-position ?
--cx-display ?

Documentation added


Presentation Ack missing?

According to the present-proof protocol, the verifier should send an ACK message to the prover.
I'm using the /present-proof/send-proposal with all the auto flags enabled at the agent which leads to the following final states:
verifier: verified
prover: presentation_sent
I would expect, that the prover ends up in a different state and I can't see anything about the ack message in the logs.

I'll take a look at this.

PR#314 fixes.


Include local nvme disks in table

Hi,
I recently looked at using L*Series VMs but I noticed that their nvme disks are not encrypted when using encryption at host, which is somewhat contrary to what I read in this document (Temp disk encryption). I am guessing you don't view the temporary nvme disks as real "temp" disks? :) If that is the case please note this somewhere in this document.
https://github.com/MicrosoftDocs/azure-docs/blame/54abcff5c453d383da8e120049dc8a98501c4700/articles/virtual-machines/lsv3-series.md#L46

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b964cc95-9074-55f1-c150-f74933aa9222
Version Independent ID: 90d48cea-1b05-d320-f939-179cd5e6b195
Content: Overview of managed disk encryption options - Azure Virtual Machines
Content Source: articles/virtual-machines/disk-encryption-overview.md
Service: virtual-machines
Sub-service: disks
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

@Klaas- Thanks for bringing this to our notice. We will look into this and provide an update accordingly.

Hi @msmbaldwin could you please take a look at this.

@SreejaBhattacharya-MSFT @msmbaldwin  any updates here?

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. If you are still experiencing a problem, you might want to move this question to Microsoft Q&A: https://learn.microsoft.com/en-us/answers/tags/94/azure-virtual-machines.  #please-close