Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
H
hidra
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
steinbac
hidra
Commits
de388ba2
Commit
de388ba2
authored
9 years ago
by
Manuela Kuhn
Browse files
Options
Downloads
Patches
Plain Diff
renamed parallelDataStream to numberOfStreams
parent
efaab83a
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
3
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
conf/dataManager.conf
+31
-31
31 additions, 31 deletions
conf/dataManager.conf
conf/dataManager_windows.conf
+59
-57
59 additions, 57 deletions
conf/dataManager_windows.conf
src/sender/DataManager.py
+145
-138
145 additions, 138 deletions
src/sender/DataManager.py
with
235 additions
and
226 deletions
conf/dataManager.conf
+
31
−
31
View file @
de388ba2
# Directory to be monitor for changes
# Inside this directory only the subdirectories "commissioning", "current" and "local" are monitored
monitoredDir
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
data
/
source
#monitoredDir
= /home/kuhnm/Arbeit/zeromq-data-transfer/data/source
#monitoredDir
= /rd
monitoredDir
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
data
/
source
#monitoredDir = /home/kuhnm/Arbeit/zeromq-data-transfer/data/source
#monitoredDir = /rd
# Target to move the files into
localTarget
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
data
/
target
#localTarget
= /home/kuhnm/Arbeit/zeromq-data-transfer/data/target
#localTarget
= /gpfs
localTarget
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
data
/
target
#localTarget = /home/kuhnm/Arbeit/zeromq-data-transfer/data/target
#localTarget = /gpfs
# Type of event detector to use (options are: inotifyx, watchdog)
#eventDetectorType
= inotifyx
eventDetectorType
=
watchdog
#eventDetectorType = inotifyx
eventDetectorType
=
watchdog
# Event type of files to be monitored (options are: IN_CLOSE_WRITE, IN_MOVED_TO, ...)
monitoredEventType
=
IN_CLOSE_WRITE
monitoredEventType
=
IN_CLOSE_WRITE
#monitoredEventType = IN_MOVED_TO
# Subdirectories of watchDir to be monitored
monitoredSubdirs
= [
"commissioning"
,
"current"
,
"local"
]
monitoredSubdirs
= [
"commissioning"
,
"current"
,
"local"
]
# The formats to be monitored, files in an other format will be be neglected
monitoredFormats
= [
".tif"
,
".cbf"
]
monitoredFormats
= [
".tif"
,
".cbf"
]
# Time (in seconds) since last modification after which a file will be seen as closed
timeTillClosed
=
2
timeTillClosed
=
2
# List of hosts allowed to connect
whitelist
= [
"localhost"
,
"zitpcx19282"
,
"zitpcx22614"
,
"haspp11eval01"
,
"it-hpc-cxi03"
]
#whitelist = ["localhost", "haspp11eval01", "it-hpc-cxi03"]
whitelist
= [
"localhost"
,
"zitpcx19282"
,
"zitpcx22614"
,
"haspp11eval01"
,
"it-hpc-cxi03"
]
#whitelist
= ["localhost", "haspp11eval01", "it-hpc-cxi03"]
# Number of parallel data streams
# if this number is modifified, the port numbers also have to be adjusted
parallelData
Streams
=
1
numberOf
Streams
=
1
# Enable ZMQ pipe into storage system (if set to False: the file is moved into the localTarget)
useDataStream
=
True
useDataStream
=
True
# Fixed host to send the data to with highest priority
fixedStreamHost
=
zitpcx19282
fixedStreamHost
=
zitpcx19282
# Fixed Port to send the data to with highest priority
fixedStreamPort
=
50100
fixedStreamPort
=
50100
# Port number to receive signals from
comPort
=
50000
comPort
=
50000
# ZMQ port to forward requests
requestFwPort
=
50002
requestFwPort
=
50002
# ZMQ port to get new requests
requestPort
=
50001
requestPort
=
50001
# ZMQ-router port which coordinates the load-balancing to the worker-processes
routerPort
=
50003
routerPort
=
50003
# ZMQ-pull-socket port which deletes/moves given files
cleanerPort
=
50004
cleanerPort
=
50004
# Chunk size of file-parts getting send via zmq
chunkSize
=
10485760
;
# = 1024*1024*10
#chunkSize
= 1073741824 ; # = 1024*1024*1024
chunkSize
=
10485760
;
# = 1024*1024*10
#chunkSize = 1073741824 ; # = 1024*1024*1024
# Path where the logfile will be created
logfilePath
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
logs
#logfilePath
= /home/kuhnm/Arbeit/zeromq-data-transfer/logs
#logfilePath
= /home/p11user/zeromq-data-transfer/logs
#logfilePath
= /home/p11user/live-viewer/logs
logfilePath
= /
space
/
projects
/
zeromq
-
data
-
transfer
/
logs
#logfilePath = /home/kuhnm/Arbeit/zeromq-data-transfer/logs
#logfilePath = /home/p11user/zeromq-data-transfer/logs
#logfilePath = /home/p11user/live-viewer/logs
# Filename used for logging
logfileName
=
dataManager
.
log
logfileName
=
dataManager
.
log
# File size before rollover in B (linux only)
logfileSize
=
10485760
;
#10
0
MB
logfileSize
=
10485760
;
#10 MB
This diff is collapsed.
Click to expand it.
conf/dataManager_windows.conf
100755 → 100644
+
59
−
57
View file @
de388ba2
# Directory to be monitor for changes
# Inside this directory only the subdirectories "commissioning", "current" and "local" are monitored
monitoredDir
=
D
:\
zeromq
-
data
-
transfer
\
data
\
source
# Target to move the files into
localTarget
=
D
:\
zeromq
-
data
-
transfer
\
data
\
target
# Type of event detector to use (options are: inotifyx, watchdog)
eventDetectorType
=
watchdog
# Event type of files to be monitored (options are: "IN_CLOSE_WRITE", "IN_MOVED_TO", ...)
monitoredEventType
=
IN_CLOSE_WRITE
# Subdirectories of watchDir to be monitored
monitoredSubdirs
= [
"commissioning"
,
"current"
,
"local"
]
# The formats to be monitored, files in an other format will be be neglected
monitoredFormats
= [
".tif"
,
".cbf"
]
# Time (in seconds) since last modification after which a file will be seen as closed
timeTillClosed
=
2
# List of hosts allowed to connect
whitelist
= [
"localhost"
,
"zitpcx19282"
,
"zitpcx22614"
,
"zitpcx17858"
]
#whitelist = ["localhost", "haspp11eval01", "it-hpc-cxi03"]
# Number of parallel data streams
# if this number is modifified, the port numbers also have to be adjusted
parallelDataStreams
=
1
# Enable ZMQ pipe into storage system (if set to False: the file is moved into the localTarget)
useDataStream
=
True
# Fixed host to send the data to with highest priority
fixedStreamHost
=
zitpcx19282
#fixedStreamHost = zitpcx17858
# Fixed Port to send the data to with highest priority
fixedStreamPort
=
50100
# Port number to receive signals from
comPort
=
50000
# ZMQ port to forward requests
requestFwPort
=
50002
# ZMQ port to get new requests
requestPort
=
50001
# ZMQ-router port which coordinates the load-balancing to the worker-processes
routerPort
=
50003
# ZMQ-pull-socket port which deletes/moves given files
cleanerPort
=
50004
# Chunk size of file-parts getting send via zmq
chunkSize
=
10485760
;
# = 1024*1024*10
#chunkSize = 1073741824 ; # = 1024*1024*1024
# Path where the logfile will be created
logfilePath
=
D
:\
zeromq
-
data
-
transfer
\
logs
# Filename used for logging
logfileName
=
dataManager
.
log
# Directory to be monitor for changes
# Inside this directory only the subdirectories "commissioning", "current" and "local" are monitored
monitoredDir
=
D
:\
zeromq
-
data
-
transfer
\
data
\
source
# Target to move the files into
localTarget
=
D
:\
zeromq
-
data
-
transfer
\
data
\
target
# Type of event detector to use (options are: inotifyx, watchdog)
#eventDetectorType = inotifyx
eventDetectorType
=
watchdog
# Event type of files to be monitored (options are: IN_CLOSE_WRITE, IN_MOVED_TO, ...)
monitoredEventType
=
IN_CLOSE_WRITE
# Subdirectories of watchDir to be monitored
monitoredSubdirs
= [
"commissioning"
,
"current"
,
"local"
]
# The formats to be monitored, files in an other format will be be neglected
monitoredFormats
= [
".tif"
,
".cbf"
]
# Time (in seconds) since last modification after which a file will be seen as closed
timeTillClosed
=
2
# List of hosts allowed to connect
whitelist
= [
"localhost"
,
"zitpcx19282"
,
"zitpcx22614"
,
"zitpcx17858"
]
# Number of parallel data streams
# if this number is modifified, the port numbers also have to be adjusted
numberOfStreams
=
1
# Enable ZMQ pipe into storage system (if set to False: the file is moved into the localTarget)
useDataStream
=
True
# Fixed host to send the data to with highest priority
fixedStreamHost
=
zitpcx19282
# Fixed Port to send the data to with highest priority
fixedStreamPort
=
50100
# Port number to receive signals from
comPort
=
50000
# ZMQ port to forward requests
requestFwPort
=
50002
# ZMQ port to get new requests
requestPort
=
50001
# ZMQ-router port which coordinates the load-balancing to the worker-processes
routerPort
=
50003
# ZMQ-pull-socket port which deletes/moves given files
cleanerPort
=
50004
# Chunk size of file-parts getting send via zmq
chunkSize
=
10485760
;
# = 1024*1024*10
#chunkSize = 1073741824 ; # = 1024*1024*1024
# Path where the logfile will be created
logfilePath
=
D
:\
zeromq
-
data
-
transfer
\
logs
# Filename used for logging
logfileName
=
dataManager
.
log
# File size before rollover in B (linux only)
logfileSize
=
10485760
;
#10 MB
This diff is collapsed.
Click to expand it.
src/sender/DataManager.py
+
145
−
138
View file @
de388ba2
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment