Diary - design & implementation thoughts

shadow-dom-element is a first implementation capable of external templates with native TEMPLATE and SLOT use by web page. As such, it could serve as façade to whole concept of native templates use for developers community. Making it simple, efficient, and sexy would serve the evangelizing purpose.
The Ananke site template demo for shadow   and light  DOM included into sources, is just a brief reference implementation which is not sufficient as bootstrap for the site and does not have a guideline.
The idea of JsonResume.org theme could be  a good starting point.

Fork-able resume theme which is based on native HTML template could attract UX creators.
site developers would be able to inject such themed resume into personal or client's site either as shadow- or light- dom-element.

Registering of such resume templates could be automated by visiting forked repos with embedded signed disclaimer. GitHub Stars or NPM stats could serve the listing priority.

Rendering JSON
🤔 most of data are passed as non-UI friendly format, novadays mostly as JSON. Native HTML template does not work with such. It injects HTML via slot. How to pass the JSON data into slot?

fetch-element approach: JSON converted into table with keys as table heading and value next to it.
slotted-element approach: inner HTML is used as template, when JSON is loaded for each key the slot with matching name is filled out.

As slotted-element is planned to be inherited form light-dom-element, its behavior could change.
Would conversion of JSON to template-able HTML be a better approach?
For simple JSON with key:value pairs there is straight answer.
`<span slot="${key}">${value}</span>`

The row of SPANs would reflect the JSON pairs and template would serve slots of interest.

PROS. simple. (is it really a good thing?)
CONS. key-value become a dom node disregarding it is used in template or not. How it could be optimized?

slot concept does not reflect the hierarchical and recurrent data structure. Can it be worked around by template+slot concept?
Can the template inclusion or recursion be used for structure and repeated slots for array relation?


Merge jobs being prioritized according to the workflow priority change

Impact of the bug
Whoever is responsible from running-closed to completed transition - WMAgent, I believe
Describe the bug
This is reported by PdmV on Monday. https://twiki.cern.ch/twiki/bin/view/CMS/CompOpsMeeting?rev=1698
List of such workflows: https://cms-pdmv.cern.ch/pmp/historical?r=RunIISummer20UL16MiniAODAPVv2,RunIISummer20UL16MiniAODv2&sortSubmittedOn=percentage&sortSubmittedOrder=-1
Here are some examples:

https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=task_EXO-RunIISummer20UL16MiniAODAPVv2-00155
https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=task_B2G-RunIISummer20UL16MiniAODAPVv2-00189
https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=task_B2G-RunIISummer20UL16MiniAODAPVv2-00236

I checked out several requests and I see that their outputs were completely produced days, and for some weeks ago. Checking running jobs, I see that there is either one LogCollect or Merge job dangling for days. I suspect again this might be related to the high number of requests currently in the system, but anyhow if output is 100% produced, why are waiting for a logcollect or merge job?
How to reproduce it
Check the requests above.
Expected behavior
If the outputs are fully produced, we should not delay the completion of the workflows.
Current issue can be fixed manually, but I'd like to understand why the system behaves this way and fix it automatically if possible.
Additional context and error message
@nsmith- @drkovalskyi  @jenimal @z4027163 FYI

Checking running jobs, I see that there is either one LogCollect or Merge job dangling for days. I suspect again this might be related to the high number of requests currently in the system, but anyhow if output is 100% produced, why are waiting for a logcollect or merge job?

I agree that waiting for a LogCollect job is not necessary.  However, I think the merge jobs need to be complete before the output is usable.  Keeping in mind that I'm not an expert, my understanding was that merge jobs take "too small" output files from individual jobs and turn them into "large enough" output files for the DM system by combining outputs together.  My understanding (again possibly incomplete) is that merge jobs take files from /store/unmerged combine them and then write them to /store/mc or /store/data, etc.  So, I think skipping that merge job would mean losing the output that job is responsible for merging.  Does that make sense?
Do we see any indication of why these merge jobs are not running?   Are they just getting crowded out by other work?  I'm not sure how prioritization with merge jobs works.

Hi,
(just observation) Not sure if this is related (or even correct and I might be wrong) - but afair - JobPrio for Merge/Cleanup/LogCollect used to be always very high value so that it runs faster than anything else.  However, I think the merge jobs need to be complete before the output is usable. Keeping in mind that I'm not an expert, my understanding was that merge jobs take "too small" output files from individual jobs and turn them into "large enough" output files for the DM system by combining outputs together. My understanding (again possibly incomplete) is that merge jobs take files from /store/unmerged combine them and then write them to /store/mc or /store/data, etc. So, I think skipping that merge job would mean losing the output that job is responsible for merging. Does that make sense?

Keeping in mind that I'm not an expert, either, according to my understanding if an output is 100% produced, then all merge jobs should already have been done and successful. So, if I am not wrong (probably I am), there should not be any running merge jobs at all if we have the output completely. On the other hand, It makes sense to me to have LogCollect jobs at this stage, but they should not hold us back this long.
Checking one of the request w/ condor_q, I do not see any job at all, though:
$ condor_q -constr 'WMAgent_RequestName =?= "pdmvserv_task_EXO-RunIISummer20UL16MiniAODAPVv2-00155__v1_T_210602_131151_4999"'

-- Schedd: vocms0284.cern.ch : <<IP_ADDRESS>:4080?... @ 07/14/21 10:57:19
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD

Total for query: 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended
Total for all users: 71809 jobs; 0 completed, 330 removed, 68624 idle, 1263 running, 1592 held, 0 suspended

gwmsmon shows 1 LogCollect: https://cms-gwmsmon.cern.ch/prodview/pdmvserv_task_EXO-RunIISummer20UL16MiniAODAPVv2-00155__v1_T_210602_131151_4999
Not sure which to believe.

Just to comment on gwmsmon - it is correct, let me explain:
Go and filter workflow here: https://cms-gwmsmon.cern.ch/prodview And if the workflow is not in the list - it is not in the system,
If you go directly to https://cms-gwmsmon.cern.ch/prodview/pdmvserv_task_EXO-RunIISummer20UL16MiniAODAPVv2-00155__v1_T_210602_131151_4999
you can see on the 'last week' diagram it was running on Friday - and after that empty graph - means no jobs are idle/running on the system (otherwise there would be red/blue lines). And also Last Update date: '7/9/2021, 11:49:28 PM'. I agree, section of Subtask still showing up a job is confusing. With a small change - this could be fixed under gwmsmon and not show jobs not on the system. gmwsmon keeps this historical information on individual workflow pages for 30days. This means if no update comes for this workflow by 8/8/2021 - it will delete that history page for good.

So we have potentially two issues here?
a) Requests whose outputs are 100% produced are still in running-closed, the original issue
b) what Justas discovered: auxiliary jobs having lower priorities than other production work in the system. If this is the case, this should never happen, IMO, as it would apparently block workflows from getting done. Who should be correcting this, @amaltaro?

Before we start debugging it, I'd suggest to first figure whether it's a general issue or likely related to a sub-set of agents and/or workflows. So, first question is, are other requests getting into the completed status? According to Grafana monitoring, there seem to be requests going into completed status, so it must be something related to specific agents.
The running-closed to completed transition is likely performed by this ReqMgr2 CherryPy thread:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/ReqMgr/CherryPyThreads/StatusChangeTasks.py
executed in CMSWEB as reqmgr2-tasks instance.
Once you identify a workflow or two to be investigated, I'd suggest to:

check the status of the GQEs (if there is anything that is not in Done/Canceled/CanceledRequested status, then it does make sense that the workflow remains in running-closed. Why 100% of the output produced though, this is yet to be figured out then.
identify which agents worked on that/those requests, ssh to them and make sure all the components are healthy and that there is nothing failing for those requests (maybe JobArchiver or TaskArchiver)
perhaps, check the number of jobs created for that/those request (from within the agent). A SQL query like this should do this (update the workflow name):

SELECT wmbs_job_state.name, count(wmbs_job.id)
FROM wmbs_job
INNER JOIN wmbs_jobgroup ON
wmbs_job.jobgroup = wmbs_jobgroup.id
INNER JOIN wmbs_subscription ON
wmbs_jobgroup.subscription = wmbs_subscription.id
INNER JOIN wmbs_sub_types ON
wmbs_subscription.subtype = wmbs_sub_types.id
INNER JOIN wmbs_job_state ON
wmbs_job.state = wmbs_job_state.id
INNER JOIN wmbs_workflow ON
wmbs_subscription.workflow = wmbs_workflow.id
WHERE wmbs_workflow.name = 'pdmvserv_task_HIG-RunIIWinter15wmLHE-00561__v1_T_151207_101950_7247'
GROUP BY wmbs_job_state.name;

something else that might be needed is to check which filesets are still open, or which subscriptions are not yet finished. A query like (update the workflow name in the query):

SELECT wmbs_subscription.id AS subId, wmbs_workflow.task AS wfName, wmbs_workflow.injected As wfInj, wmbs_fileset.name AS fsetName,
  wmbs_fileset.open AS fsetOpen, wmbs_subscription.finished AS subFin, wmbs_sub_types.name AS subType
  FROM wmbs_subscription
    INNER JOIN wmbs_sub_types ON wmbs_subscription.subtype = wmbs_sub_types.id
    INNER JOIN wmbs_workflow ON wmbs_subscription.workflow = wmbs_workflow.id
    INNER JOIN wmbs_fileset ON wmbs_subscription.fileset = wmbs_fileset.id
  WHERE wmbs_workflow.name='sagarwal_task_TOP-RunIIFall18wmLHEGS-00109__v1_T_181105_200726_3792'
    AND wmbs_fileset.open = 1
    AND wmbs_subscription.finished = 0
  ORDER BY fsetName;

Regarding the auxiliary jobs with lower priority. That should never happened, or at least it should never happen from the agent side. Of course, someone can always ssh to the agent and run a condor_qedit to change those...
When someone changes an active workflow priority, this is seen by JobUpdater which runs this code:
https://github.com/dmwm/WMCore/blob/master/src/python/WMComponent/JobUpdater/JobUpdaterPoller.py#L155-L166
which in the end executes this code:
https://github.com/dmwm/WMCore/blob/d66237302581b8d6b6a41295d82bb64c95a5296c/src/python/WMCore/BossAir/Plugins/SimpleCondorPlugin.py#L467
which takes into consideration the job type (task type) and it's priority factor. As one can see here:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMBS/CreateWMBSBase.py#L504-L505
Production/Processing should always - no matter the workflow priority - be lower priority than merge/logcollect/etc.
Last but not least, workflow completion does NOT wait for auxiliary jobs to be completed. Only Production/Processing/Merge/Harvesting jobs are considered for completing a workflow or not.

Thanks a lot @amaltaro  for the detailed explanation. Before diving into debugging, I made an impact analysis and I see that 210 workflows got affected by this issue. I attach the list here. I compared the LFN bases of these 210 workflows w/ the unmerged files at Caltech (one of the sites where unmerged usage is high), but I discovered that these 210 workflows are responsible for only 178 files out of 108k. Probably, completing these requests will save space on other grid sites, not much at Caltech. So, I'd suggest manually completing these requests for the sake of unmerged issue. Any objections?  I'll double check the list before the operation. So far, it looks good, though. Probably, leaving a few requests might help to debug the issue.
Commenting on item 1 of Alan's roadmap: There are WQEs in Running and so far I spotted only one agent related to this issue: vocms0284.cern.ch (I didn't check all the requests, only 7-8)
completeWorkflowsWMCoreBug#10672.txt
FYI @drkovalskyi  @nsmith-

I am hesitant to force-complete these 210 workflows due to the issue w/ force-completions https://github.com/dmwm/WMCore/issues/10675 I think we should fix these issues automatically. Manual operation does not seem very promising.

I force-completed one request to check if it will be affected by #10675  or not. If it succeeds, I'll force-complete all the workflows affected by this issue to decrease the stress on the system. https://cmsweb.cern.ch/reqmgr2/fetch?rid=cmsunified_task_EXO-RunIIFall17wmLHEGS-03554__v1_T_210628_132241_8381

Thanks @haozturk

OK it's been over 24 hrs and the workflow that was force completed is still sitting in force complete waiting for a merge job to run @todor-ivanov   This is next on your list of priority, doesn't pay to force complete if it doesn't work.

While going through the agents I just noticed an error which maybe worth mentioning in this issue here [1]. This may be somehow related with the not properly updated priorities of the merge jobs. So far the error I found in one agent only: vocms0257. Looking deeper now.
[1]
2021-07-22 22:59:10,485:140157693433600:ERROR:SimpleCondorPlugin:Failed to update JobPrio for WMAgent_SubTaskName=/pdmvserv_task_B2G-RunIISummer15wmLHEGS-04984__v1_T_210429_122736_8292/B2G-RunIISummer15wmLHEGS-04984_0/B2G-RunIISummer16DR80Premix-05239_0/B2G-RunIISummer16DR80Premix-05239_1/B2G-RunIISummer16MiniAODv3-06090_0/B2G-RunIISummer16MiniAODv3-06090_0MergeMINIAODSIMoutput
2021-07-22 22:59:10,486:140157693433600:ERROR:SimpleCondorPlugin:Unable to edit jobs matching constraint
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.4.7.patch3/sw/slc7_amd64_gcc630/cms/wmagent/1.4.7.patch3/lib/python2.7/site-packages/WMCore/BossAir/Plugins/SimpleCondorPlugin.py", line 486, in updateJobInformation
    schedd.edit(constraint, 'JobPrio', classad.Literal(newPriority))
  File "/data/srv/wmagent/v1.4.7.patch3/sw/slc7_amd64_gcc630/external/condor/8.9.7-comp/lib/python2.7/site-packages/htcondor/_lock.py", line 75, in wrapper
    rv = func(*args, **kwargs)
RuntimeError: Unable to edit jobs matching constraint

the workflow Hasan tried to force complete is still stuck in force complete with the same job in status queued on
vocms0284.cern.ch
It should be noted that 284 has been in this mode for several days:
agent: vocms0284.cern.ch (1.4.7.patch2)
agent last updated: 2021/7/23 (Fri) 19:24:22 UTC : 0 h 2 m
data last updated: 1 h 19 m
status: Data collect error;
team: production
data collect error: (28, 'Operation timed out after 270001 milliseconds with 0 bytes received')
a complete restart of the agent did nothing

@haozturk I believe this problem was caused by the same issue reported and resolved in https://github.com/dmwm/WMCore/issues/10675
I'm keeping this open because I want to check the job priority (possible) issue that has been reported here as well.

Thanks Alan - yes, I see that workflows that I reported in the issue description are moving. Let's keep it couple of hours open, then I can check if there is workflow left which was affected by this issue. If not, we can close the issue.

@amaltaro are we sure that this is only related to vocms0284. Please check the following request: https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=task_SMP-RunIISummer20UL18NanoAODv9-00048 It's 100% for 2 days, but not handled by that agent.

It has WQEs in Running state in both submit7 and submit8:
https://cmsweb.cern.ch/couchdb/workqueue/_design/WorkQueue/_rewrite/elementsInfo?request=pdmvserv_task_SMP-RunIISummer20UL18NanoAODv9-00048__v1_T_210720_100722_78
and those agents do not have the same problem as vocms0284 had. I will have a deeper look though to make sure those agents are working properly as well.

Yes, I've seen those WQEs, but it did not make sense to me to have WQEs in Running state while the output is 100%

@haozturk Hasan, here is a likely reason for these workflows taking longer than expected to go to completed:
https://github.com/dmwm/WMCore/issues/8477#issuecomment-887411403
Note that this is not a permanent problem, just that specific thread under-performing. I'm monitoring those agents for the next few hours to evaluate whether the issue 8477 will need to be re-prioritized or not.

@haozturk I'm changing the title of this GH issue because the main root cause has been fixed. The second issue is mentioned above, with an under-performing agent thread that has its own issue to be addressed in #8477
Thanks to @juztas , I can confirm that indeed Merge jobs are being prioritized after the workflow absolute priority number, without taking into consideration the task priority factor (job type), as just queried in submit7 [&]. I think this is a serious issue, which can lead to a) data loss; and b) workflow completion delay; so I'd like to address this merge issue in this GH issue.
[&]
[cmsdataops@cmsgwms-submit7 current]$ condor_q -const 'CMS_JobType =?= "Merge" && JobStatus =?= 1' -af:h WMAgent_SubTaskName JobPrio > alan
[cmsdataops@cmsgwms-submit7 current]$ head alan
WMAgent_SubTaskName                                                                                JobPrio
/pdmvserv_Run2018A_EGamma_UL2018_MiniAODv2_NanoAODv9_210726_092815_5804/DataProcessing/DataProcessingMergeNANOEDMAODoutput                                                                                112204
/pdmvserv_task_BTV-RunIISummer20UL17NanoAODv9-00018__v1_T_210718_184611_6255/BTV-RunIISummer20UL17NanoAODv9-00018_0/BTV-RunIISummer20UL17NanoAODv9-00018_0MergeNANOEDMAODSIMoutput                                                                                110114
/pdmvserv_Run2018A_EGamma_UL2018_MiniAODv2_NanoAODv9_210726_092815_5804/DataProcessing/DataProcessingMergeNANOEDMAODoutput                                                                                112204
/pdmvserv_task_B2G-RunIISummer15wmLHEGS-04964__v1_T_210429_084712_982/B2G-RunIISummer15wmLHEGS-04964_0/B2G-RunIISummer16DR80Premix-05187_0/B2G-RunIISummer16DR80Premix-05187_1/B2G-RunIISummer16MiniAODv3-06039_0/B2G-RunIISummer16MiniAODv3-06039_0MergeMINIAODSIMoutput                                                                                110009
/pdmvserv_task_B2G-RunIISummer15wmLHEGS-04962__v1_T_210429_084655_4123/B2G-RunIISummer15wmLHEGS-04962_0/B2G-RunIISummer16DR80Premix-05191_0/B2G-RunIISummer16DR80Premix-05191_1/B2G-RunIISummer16MiniAODv3-06043_0/B2G-RunIISummer16MiniAODv3-06043_0MergeMINIAODSIMoutput                                                                                110009
/pdmvserv_task_B2G-RunIISummer20UL17NanoAODv9-00014__v1_T_210717_141920_7421/B2G-RunIISummer20UL17NanoAODv9-00014_0/B2G-RunIISummer20UL17NanoAODv9-00014_0MergeNANOEDMAODSIMoutput                                                                                110114
/pdmvserv_Run2016F_BTagMu_UL2016_MiniAODv2_NanoAODv9_210726_165149_7078/DataProcessing/DataProcessingMergeNANOEDMAODoutput                                                                                112204

while we work on this Merge priority update issue, I think we should also take the opportunity to make a feature change, as requested in this GH issue: https://github.com/dmwm/WMCore/issues/10445
this would be a very good improvement.

Well, even though Merge jobs are prioritized the same as the Production/Processing jobs, it looks like they still manage to get slots pretty quickly. I ran condor_qedit in all our currently active agents, and more than half of them did not have any jobs to be updated:
cmst1@vocms0257:/data/srv/wmagent/current $ condor_qedit -const 'CMS_JobType =?= "Merge" && JobStatus =?= 1' JobPrio 4000000
Failed to set attribute "JobPrio" by constraint: ( (CMS_JobType =?= "Merge" && JobStatus =?= 1) )
Queue transaction failed.  No attributes were set.

while others had only a handful of jobs to be updated.

As chatted with @khurtado , he will probably manage to work on this issue today/tomorrow. Kenyi, please consider the suggestion made on this GH issue (and move it to Q3 if you start covering it, please): https://github.com/dmwm/WMCore/issues/10445

@khurtado haven't we fixed this issue with https://github.com/dmwm/WMCore/pull/10760 ?
If so, could you please have a last look at the pool and check whether Merge jobs now have a meaningful priority (>=40M)? If things look okay in the condor pool, I'd suggest to close this issue.

@amaltaro I queried the the global pool just now and did not find any Merge job with a JobPrio below 40M
[khurtado@vocms0254 ~]$ condor_q -global -constraint 'CMS_JobType == "Merge"' -af:h JobPrio | sort  | uniq -c
      4 40085025
      2 40085029
      7 40085030
<PHONE_NUMBER>1
      3 40085032
     20 40085034
      5 40110000
      1 40110003
     46 40180002
      2 40180014
      3 40200000
      2 40360008
      1 40444800


Toggle buttons???

is there Toggle buttons available????

Switches and radios are provided. Please check the documentation for more details.

is it really so hard to click through the documentation?
http://www.getmdl.io/components/index.html#toggles-section/switch

@Garbee you were a little bit faster ;-)

we need Toggle Button with groups...

You can build grouping yourself. Unless there is a part of the material design spec I am unaware of.

@Garbee I think @mrfaizanhaider means this:
https://material.io/guidelines/components/buttons.html#buttons-toggle-buttons


Document use of X-Original-URL.

Resolves #90

@dougwilson Do you have the time for this review?

I consider this PR stale.


Broken link to API reference

The link to the API reference. Looking around for a few seconds I didn't find what should replace it, so maybe it should be removed. 🎈

@wbhuberIBM Could you assign this to Nick Lange on the documentation team please? Thanks!

Link is in the readme.md
Link should be https://console.bluemix.net/docs/services/cloud-object-storage/api-reference/about-api.html
I need permissions to this branch to make the update

Tracking here: CSAFE-41308

@vanessawrites - can you make the changes in stage env, once its in stage I can push it out to prod - thanks

@wbhuberIBM - Would you be able to make this change in readme file in ibm-cos-sdk-python repo?

@vijuhullur yes we're releasing the fix in the upcoming patch.

@mtn this was fixed in release 2.1.3. Please close the issue.


Amplify Frontend Hosting, Deploy fail without error message

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
d1xmjf241aer93
Region
eu-west-1
Amplify Console feature
No response
Describe the bug
Amplify Frontend Deployment fails without any error message.
Only log avalible in the deploy step is:  2021-06-21T19:45:13 [INFO]: Beginning deployment for application d1xmjf241aer93, branch:web-test, buildId<PHONE_NUMBER>

Expected behavior
Deploy should return an error message /  or deploy should work
Reproduction steps

Setup basic starter template for next js
Deploy via Amplify console

Build Settings
2021-06-21T19:44:28.329Z [INFO]: # Starting phase: preBuild
                                 # Executing command: npm ci
2021-06-21T19:44:34.916Z [INFO]: ><EMAIL_ADDRESS>postinstall /codebuild/output/src822342047/src/#########/#########/node_modules/core-js-pure
                                 > node -e "try{require('./postinstall')}catch(e){}"
2021-06-21T19:44:34.985Z [INFO]: added 709 packages in 6.347s
2021-06-21T19:44:34.993Z [INFO]: # Completed phase: preBuild
                                 # Starting phase: build
2021-06-21T19:44:34.993Z [INFO]: # Executing command: npm run build
2021-06-21T19:44:35.149Z [INFO]: ><EMAIL_ADDRESS>build /codebuild/output/src822342047/src/########/#########
                                 > next build && next export
2021-06-21T19:44:35.519Z [INFO]: info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
2021-06-21T19:44:35.626Z [INFO]: warn  - No build cache found. Please configure build caching for faster rebuilds. Read more: https://nextjs.org/docs/messages/no-cache
2021-06-21T19:44:35.644Z [INFO]: Attention: Next.js now collects completely anonymous telemetry regarding usage.
2021-06-21T19:44:35.645Z [INFO]: This information is used to shape Next.js' roadmap and prioritize features.
                                 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
                                 https://nextjs.org/telemetry
2021-06-21T19:44:35.676Z [INFO]: info  - Checking validity of types...
2021-06-21T19:44:36.687Z [INFO]: info  - Creating an optimized production build...
2021-06-21T19:45:00.295Z [INFO]: info  - Compiled successfully
2021-06-21T19:45:00.297Z [INFO]: info  - Collecting page data...
2021-06-21T19:45:01.338Z [INFO]: info  - Generating static pages (0/5)
2021-06-21T19:45:02.337Z [INFO]: {} null
2021-06-21T19:45:02.351Z [INFO]: info  - Generating static pages (1/5)
2021-06-21T19:45:02.365Z [INFO]: info  - Generating static pages (2/5)
2021-06-21T19:45:02.376Z [INFO]: info  - Generating static pages (3/5)
2021-06-21T19:45:02.397Z [INFO]: info  - Generating static pages (5/5)
2021-06-21T19:45:02.397Z [INFO]: info  - Finalizing page optimization...
2021-06-21T19:45:02.400Z [INFO]:
2021-06-21T19:45:02.406Z [INFO]: Page                                Size     First Load JS
                                 ┌ ○ /                               14.8 kB         359 kB
                                 ├   └ css/5e9bff0df9120172fae9.css  496 B
                                 ├   /_app                           0 B             344 kB
                                 ├ ○ /404                            3.17 kB         347 kB
                                 ├ ○ /profile                        16.1 kB         360 kB
                                 └ ○ /protected-client               569 B           345 kB
                                 + First Load JS shared by all       344 kB
                                 ├ chunks/framework.64eb71.js      42 kB
                                 ├ chunks/main.8f141f.js           20.2 kB
                                 ├ chunks/pages/_app.628f87.js     280 kB
                                 ├ chunks/webpack.058b47.js        2.37 kB
                                 └ css/07a88b48765a1b1459e3.css    621 B
2021-06-21T19:45:02.407Z [INFO]: λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
                                 ○  (Static)  automatically rendered as static HTML (uses no initial props)
                                 ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
                                 (ISR)     incremental static regeneration (uses revalidate in getStaticProps)
2021-06-21T19:45:02.759Z [INFO]: info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
2021-06-21T19:45:02.958Z [INFO]: info  - using build directory: /codebuild/output/src822342047/src/#########/#########/.next
2021-06-21T19:45:02.960Z [INFO]: info  - Copying "static build" directory
2021-06-21T19:45:02.964Z [INFO]: info  - No "exportPathMap" found in "next.config.js". Generating map from "./pages"
2021-06-21T19:45:02.965Z [INFO]: info  - Launching 3 workers
2021-06-21T19:45:02.965Z [INFO]: info  - Exporting (0/4)
2021-06-21T19:45:02.966Z [INFO]: info  - Copying "public" directory
2021-06-21T19:45:03.833Z [INFO]: info  - Exporting (1/4)
2021-06-21T19:45:03.850Z [INFO]: info  - Exporting (2/4)
2021-06-21T19:45:03.886Z [INFO]: info  - Exporting (3/4)
2021-06-21T19:45:03.954Z [INFO]: info  - Exporting (4/4)
2021-06-21T19:45:03.955Z [INFO]: Export successful. Files written to /codebuild/output/src822342047/src/#########/#########/out
2021-06-21T19:45:03.965Z [INFO]: # Completed phase: build
2021-06-21T19:45:04.009Z [INFO]: ## Build completed successfully
2021-06-21T19:45:04.010Z [INFO]: # Starting caching...
2021-06-21T19:45:04.094Z [INFO]: # Creating cache artifact...
2021-06-21T19:45:04.098Z [INFO]: # Cache artifact is: 0MB
2021-06-21T19:45:04.098Z [INFO]: # Uploading cache artifact...
2021-06-21T19:45:04.171Z [INFO]: # Caching completed
2021-06-21T19:45:04.178Z [INFO]: No custom headers found.
2021-06-21T19:45:04.192Z [INFO]: # Starting build artifact upload process...
2021-06-21T19:45:04.433Z [INFO]: # Build artifact is: 1MB
2021-06-21T19:45:04.433Z [INFO]: # Uploading build artifact '__artifacts.zip'...
2021-06-21T19:45:04.462Z [INFO]: # Build artifact is: 1MB
2021-06-21T19:45:04.462Z [INFO]: # Uploading build artifact '__artifactsHash.zip'...
2021-06-21T19:45:04.670Z [INFO]: # Build artifact upload completed
2021-06-21T19:45:04.670Z [INFO]: # Starting environment caching...
2021-06-21T19:45:04.671Z [INFO]: # Uploading environment cache artifact...
2021-06-21T19:45:04.747Z [INFO]: # Environment caching completed
Terminating logging...

Additional information
Next JS, Static build with next build && next export

I also have a hard time tracking down any logs from codebuild, I'm assuming codebuild service is used. I can see my CloudFront distribution but nothing else.
In addition to my bug report above I also wonder:

Is amplify frontend deployment posiable to configure with code (yaml or json files in amplify stack) or do i have to use the console?
Is CodeBuild used? and in if that is the case, can i somehow find my build somewhere, so i can get some detailed output?
Is the FrontEnd Deployment available via CloudFormation?  Can i view the stack somehow?

I managed to solve my issue with the help of this part in the FAQ. So I'm closing this now, but i would stil love to get some info regarding my comment above:

I also have a hard time tracking down any logs from codebuild, I'm assuming codebuild service is used. I can see my CloudFront distribution but nothing else.
In addition to my bug report above I also wonder:

Is amplify frontend deployment posiable to configure with code (yaml or json files in amplify stack) or do i have to use the console?
Is CodeBuild used? and in if that is the case, can i somehow find my build somewhere, so i can get some detailed output?
Is the FrontEnd Deployment available via CloudFormation?  Can i view the stack somehow?


added necessary JEE modules for JRE >= 9

fix for https://github.com/highsource/jsonix-schema-compiler/issues/81

Bump - Would like to use this with later versions of Java without having to build it myself.

@highsource any news on this? At the moment we can't use your library with current java versions unfortunatly :(.

Thanks @kreftl !
In addition to your changes, for OpenJDK 15 to work I needed to do the following changes as well:
In pom.xml:
                                        <configuration>
-                                               <source>1.6</source>
-                                               <target>1.6</target>
+                                               <source>1.7</source>
+                                               <target>1.7</target>
                                        </configuration>

In npm/pom.xml:
-                                               <phase>generate-sources</phase>
+                                               <phase>prepare-package</phase>


Project Feedback: Peer Review

The documenation for the project was really good and the steps mentioned to run the app locally was accurate enough to help me get things running.
The functionality to help users search through their saved work was good and is working as expected along with the data visualization feature.
Although plotting the data took some time for me but the functionality works as expected.
Thanks,
Team Viyad

Thank you for your feedback. Appreciate it!


Add new game: 2048

Clone of https://github.com/gabrielecirulli/2048

Very nice, thanks @Jazeem !!
I'm on vacation at the moment but will take a look when I'm back next week.

Sure @lanctot
I'm curious to see if any algorithms implemented in open_spiel would be able to reach 2048. I tried the default mcts implementation, but it was not faring better than a random player.

Hi @Jazeem,
Hmm.. MCTS performing similarly to random-- even for MCTS with random rollouts-- I'm surprised. I suspect a bug in the game implementation. How many simulations did MCTS have? And how long is the maximum length of each game?
Other algorithms: you could also try some RL algorithms like DQN. Or, expectimax with a heuristic evaluation function.

 If there are no intermediate rewards then I can see why it would be super hard for vanilla MCTS but still I would expect it to be better than random.

@lanctot yeah you are correct. I think it's because there's no intermediate rewards.
When I ran the simulation over 100 games, MCTS and random couldn't win once.
But when I reduced the winning condition to tile 128 instead of 2048, MCTS could win 100 games vs 48 games for random.

I suspect the terminal condition and reward function still need some tweaking. 2048 doesn't just end when the player reaches 2048, it keeps going forever. The score isn't win/lose, it's a large number that increases based on the number and size of the merged tiles.
Current SOTA algorithms can get scores of ~500k, which is far beyond reaching 2048.

Hi @steventrouble
I was experimenting around the terminal condition and reward function. My initial assumption was that setting the game to end at 2048 might make it easier to solve.
I've since then modified the reward system to give a reward for every new tile unlocked. But like you suggested it might be better to keep the game forever like the original and modify Rewards() as score from current action and Returns() as total score.

Hi @steventrouble
I was experimenting around the terminal condition and reward function. My initial assumption was that setting the game to end at 2048 might make it easier to solve.

I recommend we make this as consistent with the original scoring (or with other RL implementations) as possible.

I've since then modified the reward system to give a reward for every new tile unlocked. But like you suggested it might be better to keep the game forever like the original and modify Rewards() as score from current action and Returns() as total score.

I think this should be configurable by a game parameter. All games in OpenSpiel are episodic: the tests assume this, so they should end in finite time (e.g. the game simulation tests can't go on forever) . I recommend you make a flag like "max_steps" or "max_game_length" parameter that plays 2048 for a finite number of steps or a flag that ends the episode after getting 2048 as the default, and then supporting the infinite game as a non-default parameter value.

Clone of https://github.com/gabrielecirulli/2048

Can you clarify how much of the code in here is based on this implementation? If you actually copied code from that repos, we likely have to put their copyright notice in the file before we can import. (And also it'd be a nice courtesy to tag the authors to let them know, they might have some comments / preferences).

I think this should be configurable by a game parameter. All games in OpenSpiel are episodic: the tests assume this, so they should end in finite time (e.g. the game simulation tests can't go on forever) . I recommend you make a flag like "max_steps" or "max_game_length" parameter that plays 2048 for a finite number of steps or a flag that ends the episode after getting 2048 as the default, and then supporting the infinite game as a non-default parameter value.

I've modified Returns() and Rewards() to be consistent with the original 2048 scoring. Have also introduced game parameters max_game_length and max_score (Currently have set them to INT_MAX, if this can cause any for any tests, training algorithms I can change this)

Clone of https://github.com/gabrielecirulli/2048
Can you clarify how much of the code in here is based on this implementation? If you actually copied code from that repos, we likely have to put their copyright notice in the file before we can import. (And also it'd be a nice courtesy to tag the authors to let them know, they might have some comments / preferences).

Haven't actually copied any code since the original is written in Javascript and also had to make some changes to fit the LegalActions(), LegalChanceOutcomes(), DoApplyAction() architecture of open_spiel.
But I did use some of the same functions in the original repo and rewrote it in cpp. Those are BuildTraversals(), FindFarthestPosition(), TileMatchesAvailable(), PrepareTiles()
Also DoApplyAction() was written by looking to the move() function in the original repo

(And also it'd be a nice courtesy to tag the authors to let them know, they might have some comments / preferences).

That'd be great!

Hi @Jazeem, something changes on the MacOS behavior via GitHub Actions, can you apply these changes to trigger the tests again: https://github.com/deepmind/open_spiel/pull/907/files

Yeah it'll take more than just that one line, now there seems to be issues with the Jax versions .. :(
Keep an eye on https://github.com/deepmind/open_spiel/pull/907, might only be resolved tomorrow.

Keep an eye on https://github.com/deepmind/open_spiel/pull/907, might only be resolved tomorrow.

Sure. Will do 👍

Keep an eye on #907, might only be resolved tomorrow.

Sure. Will do 👍

Ok tests are passing, so you just need to update the Jax and etc. package versions (one line change in install.sh)

@Jazeem thanks, looking good! Are you done fixing things up for the moment?
I've contacted @tewalds who will review the import, would be good to pull this one in now if you're done making changes.

@lanctot
Yes, done fixing things up for now.

Hi @Jazeem, it's now been merged, but a fair bit changed from internal review. Please take a look through it let us know what you think. You can still do the almost-infinite version by setting max_tile to a very high value.

@lanctot It looks great 👍
Only thing I noticed is that if a player/agent makes a move that doesn't change anything on the board, then additional chance tiles are not added. So there could be a few inconsequential moves like that, so not sure if those should be counted in
int MaxGameLength() const override { return 2 * 2 * max_tile_; }


Fix #2474: Check conflicting top-level exports

Review by @sjrd

That's all.

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/3298/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/4285/
Test PASSed.

Updated

I manually checked that this actually fails and reports the right classes.

LGTM

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/3300/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/4288/
Test PASSed.


[Build] Build issue with modern cmake targets

Describe the bug
I'm doing library upgrades for blender and have updated both our OIIO (to <IP_ADDRESS>) and OSL (to <IP_ADDRESS>) dependencies.
Leading to an odd build issue: ever since #2957 the OpenImageIO::OpenImageIO  target inside  OpenImageIOTargets.cmake gets populated as follows (abbreviated for clarity):
set_target_properties(OpenImageIO::OpenImageIO PROPERTIES
  INTERFACE_COMPILE_DEFINITIONS "OIIO_STATIC_DEFINE=1"
  INTERFACE_INCLUDE_DIRECTORIES [correct stuff here]
  INTERFACE_LINK_LIBRARIES [correct stuff here];\$<LINK_ONLY:PNG::PNG>;\$<LINK_ONLY:ZLIB::ZLIB>;[more correct stuff here]
)

Which is causing some issues for OSL, since OpenImageIOTargets.cmake makes no attempts to locate these targets, it is fine for the ZLIB target, since OSL has a checked_find_package (ZLIB REQUIRED) in its externalpackages.cmake PNG was not quite as lucky and the OSL builds fails during the cmake generation phase, since the PNG::PNG target is unknown.
Worked around the issue by adding a small patch to add checked_find_package (PNG REQUIRED) to OSL's externalpackages.cmake and adding a -DPNG_ROOT=... to our OSL build scripts.
While it solves our immediate problem, it's a "less great" solution, given it requires OSL having to know OIIO's linkage needs
To Reproduce
Steps to reproduce the behavior:

Try building  OIIO (<IP_ADDRESS>) and OSL (to <IP_ADDRESS>) together.
That's it really.

Expected behavior

Glorious success

Evidence
While configuring OSL with CMake.
  CMake Error at src/liboslcomp/CMakeLists.txt:19 (add_library):
    Target "oslcomp" links to target "PNG::PNG" but the target was not found.
    Perhaps a find_package() call is missing for an IMPORTED target, or an
    ALIAS target is missing?
[ above text 7x per target ]
  -- Generating done
  CMake Generate step failed.  Build files cannot be regenerated correctly.

Platform information:

CMake version: 3.20.5
OIIO branch/version: OIIO (<IP_ADDRESS>) and OSL (to <IP_ADDRESS>)
OS: Win10
C++ compiler: MSVC 2017 15.9
Any non-default build flags when you build OIIO: build config available here

I don't believe that anything from PNG or ZLIB affects the public APIs, so I don't think those need to be explicitly passed to the downstream client of OIIO. So I'm not sure why those targets would end up in the exported OIIO config. But I'll look into it.

Good point, should probably have mentioned that, we link oiio statically, so downstream needs to be informed about the linktime requirements, this is was working before #2957 but it sounds that may have been pure luck :)

OK, static linkage is admitted not as thoroughly tested.
I guess that when you build OIIO, it understands that the libraries are static and so passes the targets to the downstream project... but as you point out, if the downstream project didn't already do a find_package to define the target, it wouldn't know what it means.
How does this usually work for CMake projects? What is the customary way to solve it?

I think that before #2957, we used the variables like ZLIB_LIBRARIES, so it would have embedded the expansion of those, which is the full path.
I'll have to read up a bit on the cmake side of this. I thought switching from vars to targets was the modern way, but I'm not sure how that's supposed to work for static libs. Hmmm.

I'm no CMake expert, Instinctively I want to stick a set(depname_ROOT where_ever_it_may_be) and find_package(depname) into the generated OpenImageIOTargets.cmake but I admit, my instincts are often wrong.

Also it's good to note, for now i'm perfectly happy using the workaround I found, so there's no need to rush this, I filed the ticket more as a heads up of a potential issue. Take your time on this, there is no rush.

Yeah, but I don't want that for all the dependent libraries (which do not need exposure through OIIO's public APIs) for the dynamic case. I think I have some reading to do. :-)

Is there anything that still needs to be solved here?

Fixing things without even knowing it is the best kind of fixing things!


Add basic support for ANSI terminal escape sequences

when printing the cells of the view, we look for escape codes, and
update the attributes of the cell accordingly. This is done via the
escapeIntepreter struct and methods
add a colors.go example

The code currently only supports the 8 basic colors, and bold,
underscore and reverse.

Updated the branch with a fix so that attributes carry over the new line

First of all, sorry for the late response. Thanks for the contribution!
I'll review it to see if it fits with the internals refactoring I have in progress. Basically, I want to use some kind of cell struct (containing color info) instead of rune :) So it will be easier to implement edition, buffer reading, etc.

No worries. I see that there are conflicts now, let me know when you're closer to a review, and I'll update the PR.
Thanks!

Hi @deweerdt ,
I just pushed a commit that adds support for colored text (b342d3ad9d34df3e0780428becb3b0a44a12f35c).
Basically, now contents are stored as cells (rune + fgcolor + bgcolor) and ESC sequences are processed when the data is written to the View via View.Write(). This way:

Having the color information in a cell struct makes much more easier to implement Edition modes (displacements, conversion between visual a internal buffer coordinates, etc).
When reading from the View, it is possible to read only the data without colors, or the data + colors and then output it in a given format.
Of course, users can use any external ansi-color package in combination with gocui to add colored text to the view. Of course, it is also possible to write just raw data directly.

For ESC sequences, I used your escapeInterpreter. But did some style changes. Could review those changes? On the other hand, looks like you know quite well how ANSI colors work, would it be possible to create some tests for the escapeInterpreter?
Finally, please check if everything works as expected. I also added a little example under _examples/colors.go :)
Thanks a lot for your great contribution! :)

I forgot the mandatory screenshot :)

BTW note that all this work is in the branch colors.

That sounds good, I'll take a look at b342d3a this week-end. Thanks!

Cool! Bear in mind that I pushed more commits to the branch colors (https://github.com/jroimartin/gocui/commits/colors).

The changes in  b342d3a look good to me, thanks!

Closing this #PR in favor of  b342d3a

Merged in master! Thanks! :D


Fix precision of EdgIntQad and DisVerTri

Dear Loic,
Please find those 2 fixes we found by projecting random (fuzzing) points on a z=0 disk containing some almost degenerated triangles.
I hope I didn't miss something about the usefulness of the barycentric coordinate test. If I did, I hope you could explain it to me.
Jerome

Hi Jerome,
thanks for your bug correction.
I will have some free time to check and validate it by the end of June.
Best regards,
Loïc


Fix to issue531 sleeplog ignored if missing value on first night

Fixes #531 by making sure that sleeplog is used if any (not just the first) night of sleep has a data entry in the sleeplog.

Checklist before merging:

[x] Existing tests still work (check by running the test suite, e.g. from RStudio).
[ ] Added tests (if you added functionality) or fixed existing test (if you fixed a bug).
[ ] Updated or expanded the documentation.
[x] Updated release notes in inst/NEWS.Rd with a user-readable summary. Please, include references to relevant issues or PR discussions.
[ ] Added your name to the contributors lists in the DESCRIPTION and CITATION.cff files.

Codecov Report

Merging #532 (98b368d) into master (8b9c83f) will not change coverage.
 Consider uploading reports for the commit b1a4423 to get more accurate results

@@           Coverage Diff           @@
##           master     #532   +/-   ##
=======================================
  Coverage   73.25%   73.25%
=======================================
  Files          93       93
  Lines       11256    11256
=======================================
  Hits         8246     8246
  Misses       3010     3010

Impacted Files
Coverage Δ

R/g.part4.R
73.74% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b9c83f...b1a4423. Read the comment docs.


Capture correct radio button value in a form input event

Solution implemented is to return an optional value from the target element, which for all cases except radio buttons is a Some.  For radios, only the checked element will return a Some.  This prevents the last radio in the group from always overwriting the group value.

Looks good - though I will follow this PR up with some changes that make the same fix for desktop.


Pass authi token in header not working

I am trying to pass token key in restassured.  I am seeing status code :401
Response res = given()
.header("Token","Value")
.when()
.get ("").then().
extract().response();
I tried passing key value with .header("Token", "value") but was not successful

working fine now


Question - is there a way to bypass the plugin in case of need

Hi
The plugin works great (thanks!) yet some of the devices may not get the authenticated (nuances associated with Identity Provider, especially for mobile devs) and I'd need to grant them access someway. Is there a way to bypass the plugin and offer (through cleverly crafted link, for example) direct access to user/password style of authentication (existing previously in the system)?

Same issue as #23 .  I am working on a new release to address this.

Same issue as #23 .  I am working on a new release to address this.

Added local login capability in release 1.1.0

Added local login capability in release 1.1.0

Thanks for Your help - highly appreciated!!!!
I've posted my comment (some issue to use 1.1.0) on #23 before noticing it was closed. Should we continue on this thread?

Thanks for Your help - highly appreciated!!!!
I've posted my comment (some issue to use 1.1.0) on #23 before noticing it was closed. Should we continue on this thread?

Thanks for the info on the update script.  I'll review and correct.
If you're getting a blank page then it may be a php error.  What do you logs show?
Are the database values saved in requested_authn_context and requested_authn_context_comparison after you select them and save?  For the original configuration, the requested_authn_context should be set to passwordprotectedtransport and requested_authn_context_comparison should default to exact.

Thanks for the info on the update script.  I'll review and correct.
If you're getting a blank page then it may be a php error.  What do you logs show?
Are the database values saved in requested_authn_context and requested_authn_context_comparison after you select them and save?  For the original configuration, the requested_authn_context should be set to passwordprotectedtransport and requested_authn_context_comparison should default to exact.

I was able to recreate your issue and resolve.  I've updated the 1.1.0 branch.

I was able to recreate your issue and resolve.  I've updated the 1.1.0 branch.

Thanks Derrick!
var/log/httpd/error_log shows nothing, since the correction in the database (there were some indexing problems, but they're gone since db has been supplied with missing columns)
values in aforementioned columns are stored correctly, currently they are:
requested_authn_context = PasswordProtectedTransport
requested_authn_context_comparison = exact
/files/_log/php-errors.log is a little more informative, as it contains:
[2021-01-28 16:40:32] glpiphplog.CRITICAL:   *** Uncaught Exception OneLogin\Saml2\Error: Invalid array settings: sp_cert_not_found_and_required in /var/www/html/plugins/phpsaml/lib/php-saml/srt/Saml2/Settings.php  at line 141
Backtrace :
...ins/phpsaml/lib/php-saml/src/Saml2/Auth.php:167 OneLogin\Saml2\Settings->__construct()
plugins/phpsaml/inc/phpsaml.class.php:35           OneLogin\Saml2\Auth->__construct()
plugins/phpsaml/inc/phpsaml.class.php:25           PluginPhpsamlPhpsaml::init()
plugins/phpsaml/setup.php:150                      PluginPhpsamlPhpsaml->__construct()
inc/plugin.class.php:1365                          plugin_post_init_phpsaml()
inc/plugin.class.php:231                           Plugin::doHook()
inc/includes.php:106                               Plugin->init()
index.php:55                                       include()
...which seem to fail on _loadSettingsFromArray($settings)
Is there anything I should correct still? My settings are the same as before (some are new, thou, compared to version 1.0.0:

plugin enforced = No
Strict = No
Debug = Yes
SPCert = empty
SPCertKey = empty
IP Entity ID = set correctly, the same as before
IP SSO URL = set correctly, the same as before
IP Public X509 = verified, the same as before
SAC Selected = PasswordProtectTransport
RAC = exact

Thanks Derrick!
var/log/httpd/error_log shows nothing, since the correction in the database (there were some indexing problems, but they're gone since db has been supplied with missing columns)
values in aforementioned columns are stored correctly, currently they are:
requested_authn_context = PasswordProtectedTransport
requested_authn_context_comparison = exact
/files/_log/php-errors.log is a little more informative, as it contains:
[2021-01-28 16:40:32] glpiphplog.CRITICAL:   *** Uncaught Exception OneLogin\Saml2\Error: Invalid array settings: sp_cert_not_found_and_required in /var/www/html/plugins/phpsaml/lib/php-saml/srt/Saml2/Settings.php  at line 141
Backtrace :
...ins/phpsaml/lib/php-saml/src/Saml2/Auth.php:167 OneLogin\Saml2\Settings->__construct()
plugins/phpsaml/inc/phpsaml.class.php:35           OneLogin\Saml2\Auth->__construct()
plugins/phpsaml/inc/phpsaml.class.php:25           PluginPhpsamlPhpsaml::init()
plugins/phpsaml/setup.php:150                      PluginPhpsamlPhpsaml->__construct()
inc/plugin.class.php:1365                          plugin_post_init_phpsaml()
inc/plugin.class.php:231                           Plugin::doHook()
inc/includes.php:106                               Plugin->init()
index.php:55                                       include()
...which seem to fail on _loadSettingsFromArray($settings)
Is there anything I should correct still? My settings are the same as before (some are new, thou, compared to version 1.0.0:

plugin enforced = No
Strict = No
Debug = Yes
SPCert = empty
SPCertKey = empty
IP Entity ID = set correctly, the same as before
IP SSO URL = set correctly, the same as before
IP Public X509 = verified, the same as before
SAC Selected = PasswordProtectTransport
RAC = exact

Problem was in some of the default security parameters I added to the new config settings.  I've commented out the below which will allow empty SP certs.
// Indicates whether the samlp:AuthnRequest messages sent by this SP
// will be signed.              [The Metadata of the SP will offer this info]
//'authnRequestsSigned' => true,
			// Indicates whether the <samlp:logoutRequest> messages sent by this SP
			// will be signed.
			//'logoutRequestSigned' => true,

			// Indicates whether the <samlp:logoutResponse> messages sent by this SP
			// will be signed.
			//'logoutResponseSigned' =>true,

Problem was in some of the default security parameters I added to the new config settings.  I've commented out the below which will allow empty SP certs.
// Indicates whether the samlp:AuthnRequest messages sent by this SP
// will be signed.              [The Metadata of the SP will offer this info]
//'authnRequestsSigned' => true,
			// Indicates whether the <samlp:logoutRequest> messages sent by this SP
			// will be signed.
			//'logoutRequestSigned' => true,

			// Indicates whether the <samlp:logoutResponse> messages sent by this SP
			// will be signed.
			//'logoutResponseSigned' =>true,

I also corrected the install script so the tables are updated correctly when upgrading from 1.0.0 to 1.1.0.

I also corrected the install script so the tables are updated correctly when upgrading from 1.0.0 to 1.1.0.

My pleasure to confirm - works like a charm! :) Removing the mandatory encryption on the SP side did the trick.
Thanks a lot Derrick! :) :) :) 🥇

My pleasure to confirm - works like a charm! :) Removing the mandatory encryption on the SP side did the trick.
Thanks a lot Derrick! :) :) :) 🥇


Chem: SubstructureFilter: pop-up sketcher

Some sketchers are huge, cannot be made smaller, and do not look good when used in a filter group. The idea is to only show the actual substructure in a filter (without all the toolbars etc), and show the big sketcher as a pop-up when user clicks on a picture.

This is implemented already


No loop fusion for multiple results

|x:vec[i64]|
let sum = result(
	for(x, appender[i64], |b, i, n|
		merge(b, n + 1L)
	)
);
let prod = result(
	for(x, appender[i64], |b, i, n|
		merge(b, n * 2L)
	)
);
{sum, prod}

The program above produces 2 for loops over X, one for each result. I think it should produce a single loop with 2 builders.
later edit: added the generated optimized weld code:
|x|
  {result(
    (let a=(x);for(
      fringeiter(a),
      for(
        simditer(a),
        appender[i64](len(x)),
        |b,i,n|
          merge(b,(n+1L))
      ),
      |b__1,i__1,n__1|
        merge(b__1,(n__1+1L))
    ))
  ),result(
    (let a__1=(x);for(
      fringeiter(a__1),
      for(
        simditer(a__1),
        appender[i64](len(x)),
        |b__2,i__2,n__2|
          merge(b__2,(n__2*2L))
      ),
      |b__3,i__3,n__3|
        merge(b__3,(n__3*2L))
    ))
  )}

I just found out that weld.optimization.applyExperimentalTransforms enables merge_makestruct_loops which does exactly this. So I'm going to close this.


Sharefile Storage Support Option

Sharefile has options for WebDAV and FTP
Here is the API documentation: https://api.sharefile.com/rest/
A description: https://www.citrix.com/file-sharing/file-storage.html

You could try the latest beta which has FTP support?

Thanks Nick. I did try that, but couldn't get the FTP with Sharefile to work. I will try again, maybe some coffee will help

@NicholasBoccio if you can't get the FTP to work with Sharefile then make an issue and I'll see if I can fix it. Is there a free account available?

How can I securely provide you credentials?

@NicholasBoccio you can email me at<EMAIL_ADDRESS> If you want to PGP it you'll find my key in the keyservers.

@NicholasBoccio make sure you put "rclone issue 1543" in the title - I get a lot of email about rclone!

Hello,
We use sharefile extensively as well - has there been any updates with this?

There is the FTP option and there is also now a WebDAV option which appears to be supported by sharefile - can you try those @benyanke ?

I certainly will!

Any progress on this? I would love to hear how well it works?

Over the past few years we have been using ShareFile by Citrix (https://www.citrix.com/lp/sharefile/) as well. As of lately they have changed pricing on unlimited storage to $10/mo/user. We'd wondered if Citrix would ever make a Linux tool that we could use to sync our encrypted backups—so I called them. They said that they will "most likely never make one" as they are more focused on the Windows tools and products they offer. We have tested everything they support (eg. their windows sync tool and drive mapper, ftp, ftps, webdav, and their SF API v3).
I'd like to note that our account file max size limit is set to 100GB files (for some it's 10GB but you can get this changed by contacting them). All accounts come with unlimited API hits.
After testing all of these ways we've learned a lot about limitations and complications. Using certain services, files disappear after upload and takes a while before it shows up. Everything is middle-ware/proxy and loses data on uploaded files.
Here are a few results of what we found most successful:

QNAP App (max 10GB files) https://appcenter.qnap.com/eng/qpkg/overview/HybridBackup
FTP/FTPS (max ??GB files) https://support.citrix.com/article/CTX207859
WebDAV (max 2GB files) https://support.citrix.com/article/CTX207863
Citrix SF v3 API (max size set by account limitation 10GB or 100GB files)
https://api.sharefile.com/rest/index/start.aspx
Web App (https://yourname.sharefile.com) uses the API
(max size set by account limitation 10GB or 100GB files)

On top of all this, you have 18 hours to finish each file upload before it times out and disappears (through any of these services).
Our tests have shown that FTP is the worst. Any file size would fail randomly and/or not show up in your account. Download works great, though.
WebDAV is really slow to connect to, but seems to work OK with files under 2GB. We tested this with rclone and all files would upload twice as the first one couldn't get confirmed in time because it takes a bit before the file shows up in your account. The larger the file, the longer it takes before it can be confirmed in your account.
This is where we dropped all services and focused on the direct SF API they provide (see the ShareFile REST URL above). We created a Windows CLI tool (using https://github.com/citrix/ShareFile-NET) to upload files manually through this API—and this worked greatly! We tried small, medium, and large files—all the way up to 100GB. All files were delivered directly into the account and could be confirmed. Therefore, the API supports the largest file uploads at greatest speeds while avoiding their midway proxy.
After this successful test, @D3lta made a Linux tool using NodeJS and these have the same results as the Windows CLI tool above, hooray! However, this would require a lot of additional development for it to become a functional sync tool.
With all of this being said, we'd LOVE to see rclone become integrated with Citrix ShareFile's SF v3 API.

@bobdroog thank you very much for your testing - very interesting.

With all of this being said, we'd LOVE to see rclone become integrated with Citrix ShareFile's SF v3 API.

It looks like from the above a native sharefile backend would be the best performance solution.
Is there any way of getting a free account on a sharefile?

Hi @ncw - Yes, integrating rclone with their REST API is definitely the best way to go for best performance and support for large file sizes.
They offer a 30-day trial, would that be enough time?
https://www.sharefile.com/trial
ShareFile API
https://api.sharefile.com/rest/index/start.aspx

They offer a 30-day trial, would that be enough time?
https://www.sharefile.com/trial

To keep a backend working I run daily integration tests, so I'd need permanent access to an account.

To keep a backend working I run daily integration tests, so I'd need permanent access to an account.

Let me see what I can do. Will report back!

@ncw - I called them, they said they no longer have a team that would help with these kind of requests.
The only two ways right now to get an account is to purchase one or become a Citrix SaaS referral partner. I have applied to become one (through my business) and have been approved.
I just applied for the ShareFile demo account/license a few moments ago.

Please complete the form to receive your ShareFile account for 1 year. Your Advocate ID can be found in the welcome email you received with this link. Please allow 3 business days for processing.

We have received your request. Please allow 3 business days for processing.

Will report back when I get a response from them.

Hey @ncw you've got mail!

Here is a first beta of the backend
https://beta.rclone.org/branch/v1.49.0-012-g86bdb60e-sharefile-beta/ (uploaded in 15-30 mins)
I think it is feature complete.  It passes the backend integration tests, but it probably has bugs so testing appreciated!

@ncw - You are the best! @D3lta and I will start testing right now.

@ncw - You are the best! @D3lta and I will start testing right now.

Excellent :-)  Can you email me at<EMAIL_ADDRESS>- I've lost your email address!

Here is the next beta!  It supports

multipart uploads
streaming uploads
choosing the root, eg --sharefile-root-folder-id top

It should be complete now and it passes all the integration tests.
https://beta.rclone.org/branch/v1.49.0-012-gfe64f0ef-sharefile-beta/ (uploaded in 15-30 mins)

This is fantastic! Thank you so much for being on top of it! We will start testing it with larger files (up to 100GB) now.

Thank you @ncw! We will continue to test and report any found issues for improvement.

Just asking before doing more research about it.
I noticed that when getting a file from the remote to my local computer, it takes sometimes one minute before the actual download starts. When adding the --progress flag, I can see that rclone sees the files, but is not downloading them. After some time (between 30 and 90 seconds), it starts downloading the file normally.
I don't see any log message about this. Is this happening with someone else?
I want to try with a different account, to see if it's related to this, or if they throttle the download for some reason.
Upload works normally.

@Danelem I managed to see this once for a file I hadn't downloaded for a while then the next time it was quick - as if the file was being fetched from cold storage or something like that.


How to configure ApplicationInsight to get connection string from web.config or keyvault or programmatically?

Nuget: Microsoft.ApplicationInsights; and bunch of Microsoft.AI.* started packages.
Runtime: net48
Hosting environment: Windows 10 for dev and prod will be in Azure
Application: ASP.NET MVC app .NET 4.8
What are you trying to achieve?
I am trying to setup dev, test and production environment in Azure. For that, I want to get  app insight connection string value from web.config then from keyvault.  Other suggestions also welcomed.
What have you tried so far?
I tried to create telemetryclient by supplying config value, but it didn't work. It always get connectionstring value from ApplicationInsight.config file.
Thanks

I tried to create telemetryclient by supplying config value, but it didn't work.

Need to see how you are doing this to see if this is right/wrong.
TelemetryConfiguration.Active is the recommended way to modify ConnectionString, for .NET Framework apps.


error in cifar10_cnn example

Hello,
cifar10_cnn example gets argument error. Convolutional2D constructor does not have input_shape argument.
 $ python cifar10_cnn.py
 X_train shape: (50000, 3, 32, 32)
 50000 train samples
 10000 test samples
 Traceback (most recent call last):
  File "cifar10_cnn.py", line 49, in <module>
    input_shape=(img_channels, img_rows, img_cols)))
 TypeError: __init__() got an unexpected keyword argument 'input_shape'

Your Keras looks old. Input_shape was added to all layers in an earlier commit to master. Try git cloning and 'python setup.py install' the latest
(P.S. Shoutout to LTI)

cloning and reinstalling fixed that error. now I get
 File "build/bdist.linux-x86_64/egg/keras/activations.py", line 20, in relu
 AttributeError: 'module' object has no attribute 'relu'

any ideas?

Pip install the latest Theano in the Theano git repo. This instruction was added to the readme but not the setup.py (I'm assuming in hope that this fix will not be necessary when Theano is updated)

thanks updating theano has solved my problem!


Multiturn prompts, bug fixes

Get responses from LLM based on multi turn prompts instead of single turn prompt. This provides better contextual answers.
Reduce flicker in UI when updating the transcript UI.
Reduce unnecessary processing when no updates are required for transcript UI.

Seems like there is commented code at places that could be removed. Looks, works good outside of that.
If everything continues to work well, will remove the commented code.


New-style Function for differentiable and economical backprop

This is a proposal of introducing a new style Function to Chainer. It replaces the current Function. There are two purposes in this proposal.

Support differentiable backprop (cf. #388).
Reduce the computational cost of backprop; when we only need the gradient of x for a function f(x, y), the current API does not allow to skip the gradient computation for y. This proposal also tries to solve it by improving the API design.

I wrote a design document for this proposal here: https://docs.google.com/document/d/1290BiDaep18bjL9ZHs2a6mSLOTcHzbC-OQn1R9oTrHo/edit#. The design is made so that the current implementations of Function can basically be used as is. I would like to start implementing changes to the core features and then will work on updating commonly used Functions. After that, I would like to involve other members to update all the Functions.
Any comments/questions/requests are welcome!

Do you have some idea about examples how users write code to compute the second derivative and train NN with them?

For example, the grad attribute of a Variable becomes a Variable (it might be problematic to make this behavior default; in that case, it would be ok to enable this behavior only when an option (or a config) is set). When only gradients w.r.t. few input variables are required, you can write gxs = grad((y,), xs).

I found that it is dangerous to change the type of Variable.grad from ndarray to Variable because it affects existing user code too much. It is possible to add a flag (either global or an option to methods) to switch the type, then the compatibility issue is avoided but it is confusing. It is safer to keep the type of Variable.grad (then Variable.backward() cannot support differentiable backprop) and support differentiable backprop only via the new grad() function.

For backward compatibility, how about adding another attribute, say Variable.grad_as_variable?
The interface should be designed to make it easier to implement algorithms that require second-order derivatives. For example, MAML (http://arxiv.org/abs/1703.03400) updates the parameter θ as follows:
θ'_i = θ - α ∇_θ L_i(θ)
θ <- θ - β ∇_θ Σ_i L_i(θ'_i)
where L_i is a loss function and α, β are step sizes. That is, MAML tries to minimize the sum of losses L_i, each of which is evaluated after one step of gradient descent on that loss. Do you have any idea which is better to use, grad() or Variable.grad, to implement this algorithm?

That sounds good. I will consider that design.

Fixed via #2970.


there is no audio when i play the video

Just like any other day, i start my laptop up then i start downloading some films because i was on a trip next week and there is no internet there., After i was done downloading, i check if anything was corrupted and somehow it didn't have any audio, i restart my computer and try again but it still didn't work

What program did you use to watch the videos? The built in media player in windows doesn't support modern audio codec that youtube uses, specifically opus codec, so I suggest you try using another media player. I recommend VLC

Hello @shaked6540!  There is no audio stream in the downloaded files :(

Hello @shaked6540! There is no audio stream in the downloaded files :(

Check out my other comment in this thread, use VLC and see if it helps


tshark errors silently ignored

Hi,
it looks like tshark errors are silently ignored, which leads to other problems down the line. Typically, I got a "Found no signals, are you sure  supports monitor mode?" error, first because:

I didn't have the right permission to run tshark, which I fixed
then, having run some tests in root, the dumpfile used could not be overwritten by non-root user.

In these cases, instead of getting a clear error message from the subprocess, I just got the fairly unhelpful error from the script.
I would suggest the following:

show any error from tshark as a warning. Some errors are not a problem (I do get another one, that doesn't seem to hinder the proper flow), so we may not want to abort the process on error.
use proper temporary files / directory, to avoid the type of collision I encountered
maybe to use logging, to have a simpler way to manage warning / info / debug (=verbose) stuff.

I'm happy to give it a shot and submit a PR, if it's something that is deemed valuable.
Thanks!

I would appreciate it, thats for sure. I have a project where i want to count attendance at a convention, in several buildings. I was hoping to use raspberry pi zeros to do this…

Well the core tool still works. Those are quality of life improvement. I'm more asking @schollz if he's interested in such contribution :)

Absolutely! I'll merge a PR and version up no problem


Debugger ignores some try ... except ... statements

Using "Start without debugging" the following code executes without problems, but when running with debugging the program breaks with a "No module named invalidImport" error. It presents the options "Break" or "Continue" as for all non-recoverable errors. If I choose "Continue" the program continues without any issue.
The debugger somehow seems to ignore the try ...  except ... statement in the context manager while python correctly catches the error.
Clearly PTVS debugging thus should not break here.
from contextlib import contextmanager

@contextmanager
def ignoring(*exceptions):
    try:
        yield
    except exceptions:
        pass

with ignoring(ImportError):
    import invalidImport

print "Test"

The default is to break when exception is unhandled, but determining what exactly "unhandled" means is tricky. Python doesn't have two-pass exception handling that is common to most other languages, and so there's no stage at which the stack hasn't been unwound yet, but the destination of unwinding is already known. So the only way to figure out whether the exception is going to be handled at the point where it's being thrown is to try to simulate the unwinding.
Currently, we do this by looking at the source code on AST level, matching it against current line numbers on the call stack (you can see this in GetHandledExceptionRanges here). This only handles direct name-based lookup, so code like yours, which flows the exception list through a variable, doesn't work right.
Making it work for everything would be pretty complicated. On source level, we could try to rely on our code analysis engine to figure out the types instead of just doing AST parsing, but that won't cover all cases reliably, and we don't necessarily have analysis for any file that might have an except block in it. Or we could eval the expressions in except at runtime, but they potentially have side effects, and there's no reliable way to tell whether they do or not.

Note that ignoring (or rather ignored) has been added to the stdlib.  So you may start seeing this problem more frequently in the future.
https://hg.python.org/cpython/rev/406b47c64480

We should enhance our support here to handle at least the ignored context manager.
Handling arbitrary context managers is probably too complicated unless we emit an exception map from analysis before starting debugging.

@zooba I was wondering whether there is any movement on this. This feature been in the standard library since python 3.4 as contextlib.suppress. It wound be really great if at least this could be special-cased, so that we all can start writing cleaner code. At present debugging code using suppress is next to impossible. It seems weird your PR has been available for quite some time but never made into the code base.
Is there anything users can do as a stop-gap measure?
@mrocklin Do you know when the name changed from ignored to suppress? Did the ignored version ever get shipped?

No clue.  It seems more likely that I confused the name.

@mrocklin No, you were quite right. Upon futher investigation I found that this feature was proposed and implemented in Python Issue15806 but was never shipped. It was renamed in Issue19266 from ignored to suppress and shipped in this form with python 3.4.

We have a new debug engine in the 15.8 release, if you still see this problem, please open an issue at https://github.com/Microsoft/ptvsd


Implement inline mode

Could you please elaborate a bit on the issue?

Currently you can't use as an Inline bot (like @pic or @gif). Support for that feature would be great.

I see. I'd like it too but it's a little bit trick to implement and deploy.


Feature: Special build step to include IE11 and Edge support dependencies

Something like build-web-edge that when run, packages a dist web with the required TextEncoder dependency to support IE11 and Edge.
This way developers who need to support IE11 and Edge can remove an external dependency.

Microsoft dumped IE. Now they're dumping Edge.


Add metadata stepup signing

The metadta stepup signing needed to be implmented. To make sure
the metadata xml stays the same some adjustments were needed
to support the stepup metadata because not all fields should be shown.
The fields shouldn't be shown because in case of stepup only the ACS
location and the enityId is relevant. So requested attributes, UiInfo and
organization information isn't relevant and should therefore be omitted.

Merged back with: https://github.com/OpenConext/OpenConext-engineblock/pull/788


frontend: Fix deletion of cluster

There was no option to delete stateless clusters. Since they are also dynamic clusters and stored in frontend, the call to backend was useless. They had to be removed from indexDB.
Adds a confirm dialog to delete clusters
Fixes: #1780, #1781
How to test

Run frontend make run-frontend
Run backend make run-backend
Run app npm run dev-only-app

Testing

[x] Test for stateless clusters
[x] Test for dynamically loaded clusters

note: The Build container and test job failed because of that flaky test that was fixed in main, so the job was restarted.


v0.9.2: Splitter: Make nc.pdf smaller for code along session 1

The example medical plan in session 1 code along is 105 pages long. It takes a long time to process during presentation. Reduce the file (using ghostscript) to 12 pages, including the last page which has the foreign character sets.

done.


missing dependencies

after installing the library with the npm i ng-opengallery command the console showed this error:
PS > ng serve
Your global Angular CLI version (12.1.1) is greater than your local version (10.0.4). The local Angular CLI version is used.

To disable this warning use "ng config -g cli.warnings.versionMismatch false".
chunk {main} main.js, main.js.map (main) 2.04 kB [initial] [rendered]
chunk {polyfills} polyfills.js, polyfills.js.map (polyfills) 699 bytes [initial] [rendered]
chunk {runtime} runtime.js, runtime.js.map (runtime) 6.15 kB [entry] [rendered]
chunk {scripts} scripts.js, scripts.js.map (scripts) 202 kB [entry] [rendered]
chunk {styles} styles.js, styles.js.map (styles) 605 kB [initial] [rendered]
chunk {vendor} vendor.js, vendor.js.map (vendor) 341 kB [initial] [rendered]
Date: 2021-07-16T12:08:51.743Z - Hash: c571466cc5b65a68a7b9 - Time: 10680ms

ERROR in The target entry-point "ng-opengallery" has missing dependencies:
 - @thisissoon/angular-inviewport

I solved it by installing the missing package: https://www.npmjs.com/package/@thisissoon/angular-inviewport

I will look into this when I have time and fix it. Weird that no one else has had the problem or reported it, but thanks anyway !

Should be fixed now, changes are available through npm, can you confirm by updating to v0.0.6 ?


ecc: add the support for ECC

update generate_private_key to support ECC.  Move key types to constant.  Update UI for the new key types

Closes #1075


[BUG] View Inferencer not rendering details

Describe the bug
View Item Inferencer throws the error categoryIsLoading isn't defined on a preview app created using refine.new.
Settings:

Material UI, also on Chakra UI
NestJSX Cruds
Vite JS

Steps To Reproduce

Go to https://refine.new/preview/4c23df03-c08d-45d6-b07d-e69cc85c4297
Click on the view icon on any row in Blog Posts

Error
An error occured while rendering the generated component.You can check the generated code from the below "Show Code" button and fix the error manually.
If you think this is a bug, please report the issue at https://github.com/refinedev/refine/issues
Exception:
ReferenceError: categoryIsLoading is not defined
Expected behavior
Details should render by View inferencer
Screenshot

Desktop
No response
Mobile
No response
Additional Context
I have tried the setup on local as well, after upgrading to the latest version of refine it still appears.

Reported here as well before
https://github.com/refinedev/refine/issues/4575#issuecomment-1621345887

Hey @divyansh7924
Thank you for reporting this issue. We will fix this in the next release.


Discarding background patches

Hello.
Your work is very nice and so I'm trying to reproduce your result on camelyon 16.
From WSIs (tif) of camelyon 16, I'm cropping the patches, but the number of patches (about 10000 patches in a bag) is slightly larger than reported number of patches in your paper(about 8000 patches in a bag).
In a paper, I read that you discarded the background patches (entropy < 5).
Is part of discarding patches included in  your code? If so, where can i find that part?
Or What method did you use for discarding patches based on the entropy?
Thank you for your great work.

Please check the following code snippet, the actual implementation uses the sum of edges (after applying an edge detection filter)
https://github.com/binli123/dsmil-wsi/blob/f304bedbff7d275e052d9bae8e1e56c759f6e2cc/deepzoom_tiler.py#L69-L76
You can adjust the threshold by passing the argument --background_t to deepzoom_tiler.py.

May I ask whether the threshold you set, i.e., 0.25, is universal for all datasets, or it is just suitable for your experiment two datasets (TCGA and Camelyon16).
Thank you!

Please check the following code snippet, the actual implementation uses the sum of edges (after applying an edge detection filter)
https://github.com/binli123/dsmil-wsi/blob/f304bedbff7d275e052d9bae8e1e56c759f6e2cc/deepzoom_tiler.py#L69-L76
You can adjust the threshold by passing the argument --background_t to deepzoom_tiler.py.

May I ask whether the threshold you set, i.e., 0.25, is universal for all datasets, or is it just suitable for your experiment two datasets (TCGA and Camelyon16)? Thank you!

Normally 15-30 (for 8-bit images) would work for most of the datasets, You could also tweak that value and check the results to see if that gets rid of the background patches.

Hi Bin,
Thanks a lot for your active responses so far. I am using TGCA-Lung cancer dataset, with exact save SVS files as yours. But even with threshold like 30, my number of images at 5X are coming around 0.8 million, whereas yours were 0.36 million patches at 5× as you mentioned in the paper. Can you please guide me where I am doing wrong? Did you used higher threshold for Lung cancer dataset or did you used any additional filtering criteria?


number of connections

Proposal
Prometheus does not monitor its own number of connections

Can you elaborate a bit more on what you mean by "Prometheus does not monitor its own number of connections"? Specifically, what are you referring to? It would be great if you could provide more details, so that we can better understand your proposal. Thanks!


Fix upload_graph on v2

Currently, if upload_graph is specified we error out when using runner v2. This is exacerbated because we auto-opt in large graphs to this option, so some pipelines can fail without a user even enabling this option.
This fixes the problem and does the following:

Doesn't auto-opt in runner v2 pipelines
Warns and removes the upload_graph option for runner v2 pipelines if it is explicitly set.

Fixes #32159

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Example run of real streaming jobs with this option - https://github.com/apache/beam/actions/runs/10370496169

Example run of real streaming jobs with this option - https://github.com/apache/beam/actions/runs/10371405701/job/28711629663 - I added it to the streaming test suite, ran the suite, then removed it after it was clear it worked.

R: @kennknowles @liferoad

I think there's a secondary fix needed (separate PR) where we just minimize the unnecessary steps submitted in the Dataflow job request when using Runner v2 so the the request does not go over the Dataflow RPC limits (Dataflow with Runner v2 do not use such steps anyways but we just submit them for historical reasons).

I think these are the cases:

choose v2: we don't even do the v1 translation, and upload_graph is turned off / does nothing, portable pipeline staged to GCS
choose v1: upload_graph works and also we will set it automatically if necessary
leave up to the service: v1 graph follows the usual v1 path and portable pipeline staged to GCS

So there might be a corner case for a very large graph that does not choose v1 or v2. In this case, we will automatically turn on upload_graph and stage both the v1 dataflow graph and the portable pipeline to GCS. I am not certain if this works. What I said before would suggest it doesn't but I was probably wrong. Basically if the staging locations or service-side code paths have collisions then it won't work, otherwise it will.

stage both the v1 dataflow graph and the portable pipeline to GCS

Note that this will require service changes since I believe currently we use the same Dataflow job request property for both.
But agree that if we are not hitting request RPC errors with Runner v2, trimming steps in the the Dataflow job request for Runner v2 might not be a priority.

If the user explicitly requests runner v2, we already clear the steps from the REST API request.

(that was what caused the bug)


[T2A4][W09-A2]Lee Pei Xuan

work in progress (only find & sort are completed)

@pyxyyy Some comments added. Please close the PR after reading comments.
When you modify the behavior, remember to update documentation and tests as well.


Lua error when mousing over "Gear Checks" with everything enchanted

If my character has all 27/27 slots enchanted & tinkered and I mouse over "Gear Checks: 27/27", there's a lua error.
12x ...faceCharacterStatsPaneImproved/utilities.lua:5: bad argument #1 to 'pairs' (table expected, got nil)
[string "=[C]"]: in function `pairs'
[string "@CharacterStatsPaneImproved/utilities.lua"]:5: in function `tCount'
[string "@CharacterStatsPaneImproved/newstats.lua"]:431: in function <...rfaceCharacterStatsPaneImproved/newstats.lua:426>

Locals:
(*temporary) = nil
(*temporary) = "table expected, got nil"
 = <function> defined =[C]:-1

Thanks I'll have a fix shortly.

Ok line was wrong (previous version) but error is valid, fix incoming.


Broken Link

The "SharePoint Team Blog: Show off your style with SharePoint theming" link at the bottom of this page is broken/dead.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 24696dfe-b4a5-b16e-f75a-702584f44304
Version Independent ID: ebfe05bb-4726-c1c9-25ac-ca9b252d0708
Content: Deploy a custom theme in SharePoint
Content Source: docs/general-development/how-to-deploy-a-custom-theme-in-sharepoint.md
Service: sharepoint
Sub-service: sharepoint-framework
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

thx - removed from the page as link is broken.


Vue 3 support

Hello! Do you guys plan to add support for Vue 3.x?

There is a fork version of vue 3. You can use this library instead.
https://github.com/robinrodricks/vue3-touch-events


chore: fix failing tsconfig validator prop test

There was a proptest that failed in our test pipeline:
FAIL test/tsconfig/validator.test.ts
  ● Built-in matchers › Match.arrEq › pass

    Property failed after 63 tests
    { seed: -1990728707, path: "62:2:64", endOnFailure: true }
    Counterexample: [[[Number.NaN],[Number.NaN]]]

To fix this, I refactored the looseEqual helper. This also made some of the matcher code simpler.
Also added a few missing tests for some matchers.
In practice this is probably not very relevant, so I made this a chore.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

💚 All backports created successfully

Status
Branch
Result

✅
maintenance/v5.2

✅
maintenance/v5.3

Questions ?
Please refer to the Backport tool documentation and see the Github Action logs for details


Tag every resource in Kong that is managed by the ingress controller

Kong Ingress Controller can tag every resource that it manages in Kong's database.
This allows for cases where some resources in Kong are not managed by Kong Ingress controller, allowing users to configure some consumers and credentials via CRDs in k8s while others are instantiated via Admin API dynamically.
This also ties directly with Kong Service Mesh integration where the entire service mesh could be configured using another controller or via another mechanism outside of the Ingress Controller. In such a case, Ingress Controller should manage only entities that it has created and should not concern with entities in Kong's DB created by other means.

I saw the warning users should not to add kong resource by admin API  in the readme,but  could you please add some reasons. I am a little incomprehensible, is there any plan to fix it ?

@AnneXXB This is because kong-ingress-controller will periodically resync Kong entities (users, routes, plugins...) using its own Kubernetes Custom Resource Definitions, hence deleting everything that was declared using Kong's Admin API
Currently, the only possible workaround is to create these Kubernetes Custom Resource Definitions using the Kubernetes API instead of Kong's API, as I have been told by @hbagdi in issue 105

@julianguinard  Thank you for your answer。
1、Our team are busy with integrating kong and istio.  Do you any clear plan to fix this problem? if so, when or which version?
2、Why there is no vendor package in the source code package? Could you please upload the vendor package.There are so mang dependency packages we cannot get because of  the national-firewall. thank you .

@AnneXXB

1、Our team are busy with integrating kong and istio. Do you any clear plan to fix this problem? if so, when or which version?

This is high on the roadmap and will be fixed with the version coming after 0.4.0.
I'd like to know more why this particular feature is a blocker for integration with Istio in your use case. Are you planning to manually configure Kong or with another integration within Istio?

2、Why there is no vendor package in the source code package? Could you please upload the vendor package.There are so mang dependency packages we cannot get because of the national-firewall. thank you .

Please use dep to manage the dependencies of this project. You can use a proxy to put it inside a fork. We removed the vendor package a while ago and have not plans on adding it back. We feel that Go's dependency management is mature enough now to not vendor the dependencies inside a VCS.


get original data/slides

I am searching for a way to pull some data from all the slides, BUT as I found out, slick clones some slides, what makes it difficult.
so for instance, starting with this:

  …
  …

To get all images/slides
$('.show div.slide img')

will do, BUT after $('.show').slick() there are more slides added, which also have a slick-cloned class. So my question, if I filter all the .slide which do have are not .slick-cloned, will I get all images, or will there be more or less to be expected?

How about $('.show div.slide:not(.slick-clone) img')?


About clean lambda in loss calculation

Hi, thank you for the interesting work and its codes.
I have a question about the loss calculation in the code.
In the imagenet directory, the loss for obtaining saliency map is calculated as follows:
    loss=clean_lam * criterion (image,target)

However, in the case of others ('tiny-imagenet' and 'cifar' code),
the loss for obtaining saliency map is calculated as follows:
    loss=2*clean_lam*criterion (image,target)/num_classes

Is there any special reason to calculate the loss for generating a saliency map in a different way?

Hello!
Please note that

When clean_lam is positive, we update networks using aggregated gradients from clean and mixup data.
We used BCE for CIFAR and Tiny-ImageNet (codes borrowed from manifold mixup), and CE for ImageNet (codes borrowed from CutMix) for mixup data loss.
We fixed the saliency calculation method that we used CE for all datasets. (The kind of saliency loss functions do not matter much for the performance)

So the problem is that gradient scales between gradients from CE (saliency calculation with clean data) and gradients from BCE (mixup data) is different. We tried to rebalance gradient scales so that we can use an identical hyperparameter (clean_lam) regardless of the setting.
Hope this helps!

Thank you for the fast answer!


Question: Print settings for Atomic PETG?

Great work on this project.
I happen to have the same printer as you, Original Prusa i3 Mk 3, and figured I'd try some of that Atomic White PETG as well...
Was wondering what your printer settings looked like, working on dialing in now, seems to be under-extruding slightly for the stock PETG settings. I bumped my temp to 240deg C as well.

Here is the filament definition I'm using in Slic3r Prusa Edition for the Atomic Bright White PETG Pro: https://hastebin.com/vecapoyeti.ini


Make steps not images and accessible via css

So the Initial aim was just to add some css so i could access the numbers through css
i then dicovered a few issues with them. The first was the excessive amount
of css it generated due to having two images. The next one was that the
content was in an image, so didnt scale properly or accessible in the dom.
due to the concerns of adding extra html in, i decided to use before and after
which seems to be reasonably well supported. It's an improvement over what
was there, but maybe not perfect. it seems to be supported by a lot of assistive
technology, but not maybe it all, which is an improvement over none.
old

new

Should we do this upstream in Elements and govuk_frontend_toolkit too?

@fofr probably once we leave this one to settle in, i think we should maybe just have one implementation of it.

The way breadcrumbs was done is a good example.

Have found that this has already been done in Elements:
https://govuk-elements.herokuapp.com/icons-images/example-icons/
Using: circle circle-step

we should look at replacing out the one in static with this, but i think this one is still valid, the element one includes extra markup, this one doesn't.

@mcgoooo What's stopping this getting merged?

Closing this as we are taking a different approach.


use latest milmove-cypress image: MB-6024

Description
We have a repo circleci-docker that contains a build of cypress for our integration tests. When we update any packages in circleci-docker, we should pull the latest image of into mymove.
Reviewer Notes
Make sure I have the correct hash: https://hub.docker.com/r/milmove/circleci-docker/tags?page=1&ordering=last_updated
Setup
the integration tests should run w/o an issue on circle
Code Review Verification Steps

[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira story for this change

Messages

:book:
:link: MB-6024

  Generated by :no_entry_sign: dangerJS against bd3c4d6280fac8e188c5f211ccbf6c10f9b8e563

Messages

:book:
:link: MB-6024

  Generated by :no_entry_sign: dangerJS against bd3c4d6280fac8e188c5f211ccbf6c10f9b8e563

@duncan-truss that will teach me to multi-task...
A re-run worked and I have updated the hash.
Should I expect the hash to be the same when updating? Is that how you knew 38cc39df3ee891652c472a6c731feecc63a1253b would eventually be the hash?

@duncan-truss that will teach me to multi-task...
A re-run worked and I have updated the hash.
Should I expect the hash to be the same when updating? Is that how you knew 38cc39df3ee891652c472a6c731feecc63a1253b would eventually be the hash?

@duncan-truss that will teach me to multi-task...
A re-run worked and I have updated the hash.
Should I expect the hash to be the same when updating? Is that how you knew 38cc39df3ee891652c472a6c731feecc63a1253b would eventually be the hash?

Since you merged the code I just looked at the git commit hash to master.  That's the same hash they use to tag the docker images with.

@duncan-truss that will teach me to multi-task...
A re-run worked and I have updated the hash.
Should I expect the hash to be the same when updating? Is that how you knew 38cc39df3ee891652c472a6c731feecc63a1253b would eventually be the hash?

Since you merged the code I just looked at the git commit hash to master.  That's the same hash they use to tag the docker images with.