2021 November
-
TIL that if you don't put
type='button'
on yourbutton
tag, it will automatically submit the form when clicked.
2021 October
-
Look, am I a webdev with over 18 years experience? Yes.
Did I just spend 30 mins wondering why my file upload wasn't working because I forget to set the form as multipart? Also yes.
-
Just spent an hour trying to figure out why my Django webapp suddenly wasn't working because apparently if you accidentally delete an
__init__.py
you're gonna have a bad time.It took me a while because the issue only happened on the live server and not on local. Django only complained when it was run via WSGI and nothing was showing up in the logs.
2021 April
-
Me: Been using Python since 2008
Also me: Need to lookup how to use the filter function every time
(Note for future me:
list = filter(func, list)
2018 February
-
According to the documentation here: https://djangobook.com/syndication-feed-framework/
If link doesnβt return the domain, the syndication framework will insert the domain of the current site, according to your SITE_ID setting
However, I'm trying to generate a feed of magnet: links. The framework doesn't recognize this and attempts to append the SITE_ID, such that the links end up like this (on localhost):
<link>http://localhost:8000magnet:?xt=...</link>
Is there a way to bypass this?
2017 December
-
I have a query using MarkLogic node.js that basically boils down to something like this:
db.documents.query(qb.where(qb.collection('test'))).stream() .on('data', function(row) { console.log("Stream on data"); }) .on('end', function() { console.log("Stream on end"); }) .on('error', function(error) { console.log(error); }) ;
Now, for a certain collection we have in our database, the 'end' function doesn't fire, i.e. I never see "Stream on end" appear in the log. There's no error or anything, processing just stops. It's only for this particular collection, other collections seem fine.
If I query documents in that collection directly using other methods such as qb.value() without using qb.collection(), the end event fires correctly. But once I add qb.collection() into the mix (using qb.and), the end event doesn't fire.
I'm unsure how to debug this, as this is my first time trying to use streams in the nodejs client library. Any advice as to what I can check?
Thanks!
-
How to specify that a column in the schema should be nullable?
I tried adding a nullable attribute:
var myFirstTDE = xdmp.toJSON( { "template": { "context": "/match", "collections": ["source1"], "rows": [ { "schemaName": "soccer", "viewName": "matches", "columns": [ { "name": "id", "scalarType": "long", "val": "id", "nullable": 0 }, { "name": "document", "scalarType": "string", "val": "docUri" }, { "name": "date", "scalarType": "date", "val": "match-date" }, { "name": "league", "scalarType": "string", "val": "league" } ] } ] } } ); tde.validate( [myFirstTDE] );
But this gave me a template error:
"message": "TDE-INVALIDTEMPLATENODE: Invalid extraction template node: fn:doc('')/template/array-node('rows')/object-node()/array-node('columns')/object-node()[1]/number-node('nullable')"
For a template defined using XQuery, adding nullable to the column works:
<column> <name>ISSN</name> <scalar-type>string</scalar-type> <val>Journal/ISSN</val> <nullable>true</nullable> </column>
How to do the same thing using JS/Json?
2017 November
-
This is a follow-up to my question here: https://stackoverflow.com/questions/47449002/marklogic-template-driven-extraction-and-triples-dealing-with-array-nodes/47459250#47459250
So let's say I have a number of documents structured like this:
declareUpdate(); xdmp.documentInsert( '/test/tde.json', { content: { name:'Joe Parent', children: [ { name: 'Bob Child' }, { name: 'Sue Child' }, { name: 'Guy Child' } ] } }, {permissions : xdmp.defaultPermissions(), collections : ['test']})
I want to define a template that would extract triples from these documents defining sibling relationships between the children. For the above example, I would want to extract the following triples (the relationship is two-way):
Bob Child sibling-of Sue Child Bob Child sibling-of Guy Child Sue Child sibling-of Bob Child Sue Child sibling-of Guy Child Guy Child sibling-of Bob Child Guy Child sibling-of Sue Child
How can i set up my template to accomplish this?
Thanks!
-
I've been studying the examples here: https://docs.marklogic.com/guide/semantics/tde#id_25531
I have a set of documents that are structured with a parent name and an array of children nodes with their own names. I want to create a template that generates triples of the form "name1 is-a-parent-of name2". Here's a test I tried, with a sample of the document structure:
declareUpdate(); xdmp.documentInsert( '/test/tde.json', { content: { name:'Joe Parent', children: [ { name: 'Bob Child' }, { name: 'Sue Child' } ] } }, {permissions : xdmp.defaultPermissions(), collections : ['test']}) cts.doc('/test/tde.json') var tde = require("/MarkLogic/tde.xqy"); // Load the user template for user profile rows var template = xdmp.toJSON( { "template":{ "context":"content", "collections": [ "test" ], "triples":[ { "subject": { "val": "xs:string(name)" }, "predicate": { "val": "sem:iri('is-parent-of')" }, "object": { "val": "xs:string(children/name)" } } ] } } ); //tde.validate([template]), tde.templateInsert("/templates/test.tde", template); tde.nodeDataExtract( [cts.doc( '/test/tde.json' )] )
However, the above throws an Exception:
[javascript] TDE-EVALFAILED: tde.nodeDataExtract([cts.doc("/test/tde.json")]) -- Eval for Object='xs:string(children/name)' returns TDE-BADVALEXPRESSION: Invalid val expression: XDMP-CAST: (err:FORG0001) Invalid cast: (fn:doc("/test/tde.json")/content/array-node("children")/object-node()[1]/text("name"), fn:doc("/test/tde.json")/content/array-node("children")/object-node()[2]/text("name")) cast as xs:string?
What is the proper syntax for extracting array nodes into a triple?
2nd somewhat related question: say I also wanted to have triples of the form "child1 is-sibling-of child2". For the example above it would be "Bob Child is-sibling-of Sue Child". What would be the proper syntax for this? I'm not even sure how to begin with this one.
Is TDE even the way to go here? Or is it better to do this programmatically? i.e. on document ingestion, generate those triples inside the document directly?
(If it's relevant, the ML version being used is 9.)
-
I've been testing migrating one of our systems to Marklogic 9 and using the Optics API.
One of our functions involves grouping claims by member_id, member_name and getting the sums and counts, so I did something like this:
var results = op.fromView('test', 'claims') .groupBy(['member_id', 'member_name'], [ op.count('num_claims', 'claim_no'), op.sum('total_amount', 'claim_amount') ]) .orderBy(op.desc('total_amount')) .limit(200) .result() .toArray();
Above works fine. The results are of the form
[ { member_id: 1, member_name: 'Bob', num_claims: 10, total_amount: 500 }, ... ]
However, we also have a field "company", where each claim is filed under a different company. Basically the relevant view columns are claim_no, member_id, member_name, company, claim_amount
I would like to be able to show a column that list the different companies for which the member_id/member_name has filed claims, and how many claims for each company.
i.e. I want my results to be something like:
[ { member_id: 1, member_name: 'Bob', num_claims: 10, total_amount: 500, companies: [ { company: 'Ajax Co', num_claims: 8 }, { company: 'Side Gig', num_claims: 2 } ] }, ... ]
I tried something like this:
results = results.map((member, index, array) => { var companies = op.fromView('test', 'claims') .where(op.eq(op.col('member_id'), member.member_id)) .groupBy('company', [ op.count('num_claims', 'claim_no') ]) .result() .toArray(); member.companies = companies; return member; });
And the output seems correct, but it also executes quite slowly - almost a minute (total number of claim documents is around 120k)
In our previous ML8 implementation, we were pre-generating summary documents for each member - so retrieval was reasonably fast with the downside that whenever we got a bunch of new data, all of the summary documents had to be re-generated. I was hoping that ML9's optic API would make it easier to do the retrieval/grouping/aggregates on the fly so we wouldn't have to do that.
In theory, I could just add company to the groupBy fields, then merge the rows in the result query as needed. But the problem with that approach is that I can't guarantee I'll get the top 200 by total amount (as was my original query)
So, the question is: Is there a better way of doing this with a reasonable execution time? Or should I just stick to pre-generating the summary documents?
-
I have a Marklogic 9 project that I'm configuring with Roxy. I've been following these examples: https://github.com/marklogic-community/roxy/wiki/Adding-Custom-Build-Steps
Basically, I have a server-side JS function that I want to call after deploy content. I have something like this:
# then you would define your new method
def deploy_content # you can optionally call the original original_deploy_content # do your stuff here execute_query(%Q{ xquery version "1.0-ml"; xdmp:javascript-eval('var process = require("/ingestion/process.sjs"); process.postDeployContent();') }, :db_name => @properties["ml.app-name"] + "-content") end
The xquery being called here evaluates fine when executed via query console. But when I call ml local deploy content, I get the following error:
ERROR: 500 "Internal Server Error" ERROR: <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>500 Internal Server Error</title> <meta name="robots" content="noindex,nofollow"/> <link rel="stylesheet" href="/error.css"/> </head> <body> <span class="error"> <h1>500 Internal Server Error</h1> <dl> <dt>XDMP-MODNOTFOUND: var process = require("/ingestion/process.sjs"); process.postDeployContent(); -- Module /ingestion/process.sjs not found</dt> <dd></dd> <dt>in [anonymous], at 1:14 [javascript]</dt> <dd></dd> <dt>at 3:6, in xdmp:eval("var process = require(&quot;/ingestion/process.sjs&quot;); proce...") [javascript]</dt> <dd></dd> <dt>in /eval, at 3:6 [1.0-ml]</dt> <dd></dd> </dl> </span> </body> </html>
Why is the module not found when running via xquery from app_specific.rb?
Or... is there a better way to call a JS module function from here. Sorry, I'm not too familiar with the xquery side, so I just called a JS function instead.
-
Basically the title. The client is complaining that when he zooms in, the text labels for the nodes are quite large. Is there a way to keep the node labels at a fixed font size even when zooming in or out?
From the nodes documentation (http://visjs.org/docs/network/nodes.html), there's a scaling.label option, but it doesn't seem to work. I think this is only relevant if I'm using values to scale the nodes.
-
Does Roxy have support for deploying templates for use with Marklogic 9's Template Driven Extraction?
2017 October
-
I'm trying out the Roxy deployer. The Roxy app was created using the default app-type. I setup a new ML 9 database, and I ran "ml local bootstrap" using the default ports (8040 and 8041)
Then I setup a node application. I tried the following (sample code from https://docs.marklogic.com/jsdoc/index.html)
var marklogic = require('marklogic'); var conn = { host: '192.168.33.10', port: 8040, user: 'admin', password: 'admin', authType: 'DIGEST' } var db = marklogic.createDatabaseClient(conn); db.createCollection( '/books', {author: 'Beryl Markham'}, {author: 'WG Sebald'} ) .result(function(response) { console.log(JSON.stringify(response, null, 2)); }, function (error) { console.log(JSON.stringify(error, null, 2)); });
Running the script gave me an error like:
$ node test.js { "message": "write document list: cannot process response with 500 status", "statusCode": 500, "body": "<error:error xsi:schemaLocation=\"http://marklogic.com/xdmp/error error.xsd\" xmlns:error=\"http://marklogic.com/xdmp/error\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n <error:code>XDMP-IMPMODNS</error:code>\n <error:name>err:XQST0059</error:name>\n <error:xquery-version>1.0-ml</error:xquery-version>\n <error:message>Import module namespace mismatch</error:message>\n <error:format-string>XDMP-IMPMODNS: (err:XQST0059) Import module namespace http://marklogic.com/rest-api/endpoints/config does not match target namespace http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED of imported module /MarkLogic/rest-api/endpoints/config.xqy</error:format-string>\n <error:retryable>false</error:retryable>\n <error:expr/>\n <error:data>\n <error:datum>http://marklogic.com/rest-api/endpoints/config</error:datum>\n <error:datum>http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED</error:datum>\n <error:datum>/MarkLogic/rest-api/endpoints/config.xqy</error:datum>\n </error:data>\n <error:stack>\n <error:frame>\n <error:uri>/roxy/lib/rewriter-lib.xqy</error:uri>\n <error:line>5</error:line>\n <error:column>0</error:column>\n <error:xquery-version>1.0-ml</error:xquery-version>\n </error:frame>\n </error:stack>\n</error:error>\n" }
If I change the port to 8000 (the default appserver that inserts into Documents), the node function executes correctly as expected. I'm not sure if I need to configure anything else with the Roxy-created appserver so that it works with the node.js application.
I'm not sure where the "DELETE_IF_UNUSED" part in the error message is coming from either. There doesn't seem to be any such text in the configuration files generated by Roxy.
Edit: When accessing 192.168.33.10:8040 via the browser, I get a an xml with a similar error:
<error:error xsi:schemaLocation="http://marklogic.com/xdmp/error error.xsd" xmlns:error="http://marklogic.com/xdmp/error" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <error:code>XDMP-IMPMODNS</error:code> <error:name>err:XQST0059</error:name> <error:xquery-version>1.0-ml</error:xquery-version> <error:message>Import module namespace mismatch</error:message> <error:format-string>XDMP-IMPMODNS: (err:XQST0059) Import module namespace http://marklogic.com/rest-api/endpoints/config does not match target namespace http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED of imported module /MarkLogic/rest-api/endpoints/config.xqy</error:format-string> <error:retryable>false</error:retryable> <error:expr/> <error:data> <error:datum>http://marklogic.com/rest-api/endpoints/config</error:datum> <error:datum>http://marklogic.com/rest-api/endpoints/config_DELETE_IF_UNUSED</error:datum> <error:datum>/MarkLogic/rest-api/endpoints/config.xqy</error:datum> </error:data> <error:stack> <error:frame> <error:uri>/roxy/lib/rewriter-lib.xqy</error:uri> <error:line>5</error:line> <error:column>0</error:column> <error:xquery-version>1.0-ml</error:xquery-version> </error:frame> </error:stack> </error:error>
If it matters, MarkLogic version is 9.0-3.1. It's a fresh install too.
Any advice?
-
I'm trying to migrate one of my dev envts from ML8 to ML9. I have an import script that successfully works on the ML8 version, but there's an error when I try running it against the ML9 database. The ML9 version is 9.0.3.1. The MLCP version is 9.0.3
My MLCP options file is as follows:
import -host 192.168.33.10 -port 8041 -username admin -password admin -input_file_path d:\maroon\data\mbastest.csv -mode local -input_file_type delimited_text -uri_id ClientId -output_uri_prefix /test/records/ -output_uri_suffix .json -document_type json -transform_module /ingestion/transform.js -transform_function testTransform -transform_param test -content_encoding windows-1252 -thread_count 1
Here's the output of a test run with only 2 records in the test CSV file:
17/10/30 14:07:33 INFO contentpump.LocalJobRunner: Content type: JSON 17/10/30 14:07:33 INFO contentpump.ContentPump: Job name: local_455168344_1 17/10/30 14:07:33 INFO contentpump.FileAndDirectoryInputFormat: Total input paths to process : 1 17/10/30 14:07:38 WARN contentpump.TransformWriter: Failed document /test/records/31.json 17/10/30 14:07:38 WARN contentpump.TransformWriter: <error:format-string xmlns:error="http://marklogic.com/xdmp/error" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected QName_, expecting $end or SemiColon_</error:format-string> 17/10/30 14:07:38 WARN contentpump.TransformWriter: Failed document /test/records/32.json 17/10/30 14:07:38 WARN contentpump.TransformWriter: <error:format-string xmlns:error="http://marklogic.com/xdmp/error" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected QName_, expecting $end or SemiColon_</error:format-string> 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: completed 100% 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: com.marklogic.mapreduce.MarkLogicCounter: 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: INPUT_RECORDS: 2 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS: 2 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_COMMITTED: 0 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_FAILED: 2 17/10/30 14:07:38 INFO contentpump.LocalJobRunner: Total execution time: 5 sec
If I remove the transform params, the import works fine.
I thought it might be a parsing issue with my transform module itself, so I tried replacing it with the following example from the documentation:
// Add a property named "NEWPROP" to any JSON input document. // Otherwise, input passes through unchanged. function addProp(content, context) { const propVal = (context.transform_param == undefined) ? "UNDEFINED" : context.transform_param; if (xdmp.nodeKind(content.value) == 'document' && content.value.documentFormat == 'JSON') { // Convert input to mutable object and add new property const newDoc = content.value.toObject(); newDoc.NEWPROP = propVal; // Convert result back into a document content.value = xdmp.unquote(xdmp.quote(newDoc)); } return content; }; exports.addProp = addProp;
(Of course I changed the params in the MLCP options file accordingly)
The issue still persists even with just this test function.
Any advice?
-
I've been testing out the WebView component, but I can't seem to get it to render things.
Sample here: https://snack.expo.io/r1oje4C3-
export default class App extends Component { render() { return ( <View style={styles.container}> <Text style={styles.paragraph}> Change code in the editor and watch it change on your phone! Save to get a shareable url. You get a new url each time you save. </Text> <WebView source={{html: '<p>Here I am</p>'}} /> <WebView source={{ uri: 'http://www.google.com'}} /> </View> ); } }
When running the above example in Expo, neither of the two WebView components seem to render. What am I doing wrong?
-
I'm trying out vis.js for generating graph visualization. For every edge in my graph, I have a number that describes the strength of the connection between two nodes. I'd like to render the vis.js graph such that the nodes that have stronger relationships (higher edge values) are closer together (edge length is shorter).
I've set the relationship strength (an integer) as the "value" attribute for each edge, but this only seems to make the edge lines slightly thicker for higher values.
What options should I be looking at? I'm not sure if this is supposed to be a function of vis's physics-based stabilization.
Thanks for advice!
2017 September
-
I have an HTML page with around 10 charts generated by chart.js (so these are canvas elements). I want to be able to export the page content into a PDF file.
I've tried using jsPDF's .fromHTML function, but it doesn't seem to support exporting the canvas contents. (Either that or I'm doing it wrong). I just did something like:
$(".test").click(function() { var doc = new jsPDF() doc.fromHTML(document.getElementById("testExport")); doc.save('a4.pdf') });
Any alternative approaches would be appreciated.
-
I'm using ML8. I have a bunch of json documents in the database. Some documents have a certain property "summaryData", something like:
{ ...(other stuff)... summaryData: { count: 100, total: 10000, summaryDate: (date value) } }
However, not all documents have this property. I'd like to construct an SJS query to retrieve those documents that don't have this property defined. If it was SQL, I guess the equivalent would be something like "WHERE summaryData IS NULL"
I wasn't sure what to search for in the docs. Any advise would be helpful.
-
I'm using ML8 and Node.js. The documentation here: http://docs.marklogic.com/guide/node-dev/documents#id_68765 describes how to do conditional updates in ML using the versionId field.
But for example if I want to do a conditional update on a different field, is it possible?
My scenario is: I have JSON documents with elements assignedTo and assignDate (where assignDate is set to current date every time a new value is set to assignedTo)
Now, for my "Assign" operation, I would like to make sure that no one else has changed the assignedTo/assignDate fields between the time I read the document and when I perform the update. I don't care if other fields in the same document have been updated or not - if other fields have been updated, I can still proceed with the Assign operation (hence I cannot use the versionId approach, since that covers the whole document)
How can this be done?