Pandora Knowledge base (Project J)

بندورة (نظام الأرشفة)
استخدام عميل بندورة
:تنصيب عميل بندورة
:إعداد عميل بندورة
:إضافة قرص لعميل بندورة
:ترتيب الملفات والمجلدات

Questions

pan.do/ra client

• How to force re-upload of video files?

you can use pandora_client upload <oshash> or pandra_client upload <path/to/source> to reupload a video.

misc

• Pandora/What are the data structures used in pandora?

• What is the file structure?

• Where are the thumbnails generated?

frames are extracted as need and stored in 'data/media/oshash/frame/resolution' on the server. same goes for timelines, they are extracted per file and stored in data/media/oshash/ and combined per item and sotred in data/media/item/id/

• Is textual data associated with files or video items?

annotations are associated with items. moving files currently does not alter annotations at all, this would have to be done with a script. pandora could do more on the server here, but for tracking this is [1]

• What are the various things that can be applied to an Item (example; render timeline)
• Really, how to debug?

as admin you have a Debug menu, that contains an error log, check that for errors also from other users, besides that Chrome console (Ctrl-J on linux) is your friend. I.e. check the Network tab for api calls. On the server logs are in /var/log/pandora/*.log

• How to use the API; dummy examples please !

some examples from our api session at [2]

• What is done on the client side (js)?

pan.do/ra makes extensive use of javascript, the entire UI is done on the client side, and data is fetched from the server via the public api. the backend provides frames, videos, timelines, static resources and the api to access data.

• fixing the issue of more than 6 files know how

browsers only open 6 connections to one domain, the current video player opens all videos to support seeking. This leads to browers getting stuck. the current work around is to server videos from multiple subdomains.

• Changing the basic info of Items

its possible to store any data with an item api.get/api.edit, To use that info in search, sort, item keys have to be defined in /srv/pandora/pandora/client.jsonc. the item info page is currently customized per site and needs to be edited to display additional data, copy /srv/pandora/static/js/itemInfoView.js to /srv/pandora/static/js/itemInfoView.SITEID.js and edit it and run /srv/pandora/pandora/manage.py update_static to make your changes available outside of debug mode.

• How flexible is the layers of info?

items can have any number of key/value data. values can be strings, arrays of strings, numbers. display is limited by what you can do in your browser with it. Timed based layers are similar, currently there are 4 types of layers. map, event, string, text. map for locations, event for events in time, string can be used for keywords, tags or other short strings. text for longer texts. adding other layer types is possible but not easy. you can define any number of layers. layers can be public or private, on private layers, users only see there own annotations.

• To what objects are they connected?
• Show subtitles on the video!

thats possible if you define one layer with "isSubtitles": true (i.e. indiancine.ma) subtitle layers should not have overlapping annotations and fit on the video.

• For example, if video file is not served, how to really hunt what is going on?

videos look like this: /ITEM_ID/480p3.webm 480 is resolution, must be one from config.jsonc, 3 is part 3, item must have at least 3 parts. webm is the format, format must be enabled in config.jsonc. if you get 404, you can check in ./manage.py shell (import archive.models;archive.models.Streams.objects.filter(file__item="ITEM_ID")) if all streams are extracted, they are stored in data/media/oshash/480p.webm.

• Pandora/How to force regeneration of video derivatives؟

this needs to be done in manage.py shell right now:

import item.models
i = item.models.Item.objects.get(itemId="ITEM_ID")
for s in i.streams():
    s.extract_derivatives(rebuild=True 

• How to delete item from pandora shell?
• How do delete volume from pandora interface (Remove selected volume in volumes control menu is inactive so I can't delete unwanted volumes)?

  Currently this is not doable, a Volume cannot be deleted. never the less, the volume can be scanned (while pointing to an empty Volume) and its content would be thus reflected (deleted in this case) on the server. but the server will keep that information, it will keep a volume that has no files in it.

• What does (pandora_client scan rescan) do?

If some reason the encoding stopped, how can i trigger it again?

for f in archive.models.File.objects.filter(encoding=True): print f;f.streams.exclude(source=None).delete();f.process_stream()

Wishlist

• Localization infrastructure

[3]

• lists of clips

This will be called edits, so far we have always pushed this back since other features where more important. [4]

• relationship to the originals if accessable

you can lookup an item for a file by its oshash, one way is to open /file/oshash and it will redirect to the item. for an item you can get all files, and for files you get instances in volumes, that way you can lookup the local path.

• Pad.ma Local files

one way we might implement this is using HTML5 FileSystem api [5] Just fixing Pad.ma Local might be another option to allow using a local file in a video.

• create a textual representaion of the clip in conjuction with the originals

you can use the api to get all information for this. example to map this into a kdenlive project coming soon

• include annotations in the torrent

• Integration with Drupal's Media module as well as with Mediawiki

pan.do/ra supports oembed, so it might just work with existing plugins, otherwise more specific integration could be done via api, in js or php [6]

• A mechanism to separate the video curation process to two steps:
  1. remotely uploading a reduced version from a large file
  2. acquiring the original via an out-of-band method then integrating it into the datastore and linking it to the film's records (suggestion: identify the original by its hash)

this should work already with Volumes and instances. We are also working on scripts to make syncing easier.

• A mechanism to perform curation on one installation (like on a laptop, or a workshop) while travelling then integrate that into a master collection back home.
• As an implementation of arbitrary properties of an item: URL to obtain the original of a film, published somewhere on the Internet (like archive.org)

you can set any key/value on an item, item info would need be customized to show it.

• ability to export metadata of an item in some standard format (suggested RDF)
• Introduce some semantic querying service point
• Introduce crawling start-points in the system where information about the whole collection (in RDF) can be crawled and viewed semantically
• A SPARQL proxy as an external application; as a client to the Pandora API
• Allow encoding from multiple machines

pandora_client now support distributed encoding on multiple notes from one Volume

• Allow sparter parsing of pathes

pandora_client now supports plugins to overwrite path parsing logic. [7] Plugin for Mosireen at [8]

• How to add a key with multiple values?

In the config file, make sure the type has square brackets around it, and it the infoView file, add the key id to listkeys

• external sources

https://textb.org/t/external_url.patch/ ?

mass edit a value in items metadata

import item.models
for f in item.models.Item.objects.all():
    f.edit({"license":""})

ironing out installations

• ADEF Archive
• Mareekh
• Mosireen
• Mawred

Pandora client for distributed encoding

installation

*sudo easy_install pandora_client
*sudo easy_install requests

you might need to :

sudo apt-get install python-twisted-web

operation

• from the server side

pandora_client server

• from the client side

pandora_client client http://PandoraClientServer:8789

Update/Upgrade binaries used by pandora_client

pandora_client install_programs update