Difference between revisions of "IoTSec:Reviewer Comments"
From its-wiki.no
Josef.Noll (Talk | contribs) (→Assessment of relevance:) |
Josef.Noll (Talk | contribs) |
||
(7 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
+ | {{TOCright}} | ||
= Comments from Reviewer's to the proposal = | = Comments from Reviewer's to the proposal = | ||
The decisions regarding approval and rejection of grant proposals have been taken | The decisions regarding approval and rejection of grant proposals have been taken | ||
Line 15: | Line 16: | ||
Workshops, approach to standardization bodies and approach to European | Workshops, approach to standardization bodies and approach to European | ||
Technology Platforms. | Technology Platforms. | ||
− | : Comments: ''The main integration will take place along two lines, being (i) the simulation of security models and modules, and (ii) the integration in the test-bed by [[NR]] and the showroom of [[NCE Smart]] and [[eSmart Systems]].'' | + | : Comments: ''The main integration will take place along two lines, being (i) the simulation of security models and modules, and (ii) the integration in the test-bed by [[NR]] and the showroom of [[NCE Smart]] and [[eSmart Systems]]. Task [[IoTSec:T4.1|T4.1]] is updated to address the industrial ecosystem for security in IoT, and thus build the basis for a common understanding of research and industry.'' |
+ | : Comments: ''Industrial partner will make use of the results through the adaptation for the security centre for Smart Grids, addressed in [[IoTSec:WP4|WP4]].'' | ||
The overall plan could be improved by being more | The overall plan could be improved by being more | ||
selective and focusing on quality instead on quantity and wide coverage. Several | selective and focusing on quality instead on quantity and wide coverage. Several | ||
challenges remain without a convincing answer. | challenges remain without a convincing answer. | ||
− | : Comments: ''Thanks for the comment on quality, rather than quantity and wide coverage. | + | : Comments: ''Thanks for the comment on quality, rather than quantity and wide coverage. Our definition of quality follows the academic standards, being ''(i)'' the description in a technical note, ''(ii)'' the presentation on a peer-reviewed conference, and finally ''(iii)'' the peer-reviewed journal paper. We will first prepare the ground in a wide sense, and then use the feedback from the industrial partners and their requirements for the [[IoTSec:Security Centre|Security Centre]] to focus on specific challenges. Thus, we expect that we can provide a more focussed approach at M06 (after the second progress meeting) with a more streamlined approach influenced by Industry.'' |
Prosjektet vurderes til et "meget godt" forskerprosjekt. | Prosjektet vurderes til et "meget godt" forskerprosjekt. | ||
Line 38: | Line 40: | ||
nettverk blant partnerne. Hvilke nye internasjonaliseringsaktiviteter prosjektet | nettverk blant partnerne. Hvilke nye internasjonaliseringsaktiviteter prosjektet | ||
skal bidra med sies det lite om og det er heller ikke dokumentert at det inngår noe internasjonalt samarbeid. | skal bidra med sies det lite om og det er heller ikke dokumentert at det inngår noe internasjonalt samarbeid. | ||
− | : Comments: ''Here we have to be more precise, on how to build the international network. It includes a cross-European network for IoTSec, with a prototype network at Mondragon University ([[MGEP]]), an exchange of key personnel both ways, ... Regarding the European Technology platforms and collaborations, DG14 Energy invited us to the ICT 2015 conference to join a Network Session on Data Management, initiated by our collaboration project H2020 Empower.'' | + | : Comments: ''Here we have to be more precise, on how to build the international network. It includes a cross-European network for IoTSec, with e.g. a prototype network at Mondragon University ([[MGEP]]), IoTSec models with Viktoria University, an IoTSec course with University Tor Vergata, and the exchange of key personnel both ways, ... Regarding the European Technology platforms and collaborations, DG14 Energy invited us to the ICT 2015 conference to join a Network Session on Data Management, initiated by our collaboration project H2020 Empower.'' |
Prosjektets bidrag til utvikling av nye innovasjoner og samfunnsnytte forventes å | Prosjektets bidrag til utvikling av nye innovasjoner og samfunnsnytte forventes å | ||
Line 45: | Line 47: | ||
sikre at forskningsresultatene raskere finner anvendelser til nytte for samfunnet. | sikre at forskningsresultatene raskere finner anvendelser til nytte for samfunnet. | ||
− | [[Media:IoTSec-ReviewerComments_247627.pdf]] | + | = Detailed Review = |
+ | == Scientific Merits == | ||
+ | The project considers system description, security modelling, evaluation and industrial applications in the context of | ||
+ | secure applications for smart grids scenarios using IoT technologies. The research proposed will produce new | ||
+ | techniques, mechanisms and methodologies, but the proposal lacks details regarding how those aforementioned | ||
+ | results will be integrated. Overall, the focus seems to be on more applied research that is relevant to industry. The | ||
+ | project will complement an existing collaboration with a cluster of projects. The approach of "semantic provability" | ||
+ | is promising. The a priori knowledge of the project about the state-of-the-art is well documented and the scientific | ||
+ | basis is realistic. | ||
+ | |||
+ | == Project manager and project group == | ||
+ | The consortium is composed of a good combination of research and industrial partners. Actually, they conform a | ||
+ | very representative group for the particular scope of the proposal, but with limited presence in the top scientific | ||
+ | security venues. | ||
+ | : Comment: ''Through the aforementioned approach of scientific quality we will address top scientific venues. '' | ||
+ | |||
+ | The project manager is active, productive and has experience from several national and EU | ||
+ | projects with strong industrial background, but his track record in security, privacy and dependability is rather weak. | ||
+ | Overall his citation profile could have been stronger. | ||
+ | : Comment: ''The project leader was leader of the JU Artemis pSHIELD project, and industrial applicability leader of the JU Artemis nSHIELD project. Through the projects the methodology of measurable security, privacy and dependability was established. Project leadership and industrial applicability had a higher priority than academic publications.'' | ||
+ | |||
+ | International collaboration within the scope of the project is | ||
+ | foreseen due to previous and current involvement of project partners in research relationships, at different levels | ||
+ | (projects, publications, ...) with European universities and companies | ||
+ | |||
+ | == Implementation plan and resources == | ||
+ | The project has very detailed workpackages, tasks and milestones, showing an overall clear structure. The roles | ||
+ | and responsibilities of the partners are well-defined. Also, resources requested are reasonable considering the size | ||
+ | of the consortium and duration of the project. However, the implementation plan lacks to ''clearly show how | ||
+ | industrial partners will make use of project results.'' | ||
+ | :Comment: ''see above, updated definition of Task [[IoTSec:T4.1|T4.1]] and [[IoTSec:WP4|WP4]]'' | ||
+ | |||
+ | ''The quantity and diversity of approaches produces some | ||
+ | concern about the focus and integration.'' | ||
+ | :Comment: ''The quantity of approaches was introduced to define the basis on top of which IoTSec will define the focus. The focus of the project, applicability in a Smart Grid Security Centre, is highlighted through an updated task description in [[IoTSec:WP4|WP4]]. A detailed task and workpackage description will address which approaches are taken into consideration.'' | ||
+ | |||
+ | It is not clear that there is a ''validation planned with real users'', even if | ||
+ | usability seems to be one of the goals. | ||
+ | :Comment: ''Validation with real users is addressed in two ways, being ''(i)'' the simulations in the eSmart show room and ''(ii)'' the applicability in the Smart Grid Security Centre.'' | ||
+ | |||
+ | Overall, ''it could have been made more clear what the research outcomes | ||
+ | will be'' beyond research papers and project proposals. | ||
+ | :Comment: ''The research outcomes are better addressed through the extended descriptions of the tasks and the workpackages, including detailed objectives and expected outcome''. | ||
+ | |||
+ | == Dissemination and communication of results == | ||
+ | The dissemination plan is very solid, broad and detailed as it targets, in a realistic way, not only typical publication | ||
+ | activities but also organization of specific Workshops, approach to European Technology Platforms, and approach | ||
+ | to standardization bodies, which is essential in this area. It should also involve the top security conferences that | ||
+ | guarantee the highest visibility in the international community. The proposal has measurable outcomes in terms of | ||
+ | dissemination, what is valuable. | ||
+ | |||
+ | == Overall assessment == | ||
+ | This project addresses a timely and challenging topic. The team has a very good background in the area and builds | ||
+ | on existing collaborations, bringing together academia and industry. The scientific expertise is very good but not | ||
+ | excellent. The project lacks details regarding how research results will be integrated, and how industrial partners | ||
+ | will make use of project results. The dissemination plan is very solid as it targets typical publication activities but | ||
+ | also organization of specific Workshops, approach to standardization bodies and approach to European | ||
+ | Technology Platforms. The overall plan could be improved by being more selective and focusing on quality instead | ||
+ | on quantity and wide coverage. Several challenges remain without a convincing answer. | ||
+ | |||
+ | all taken from the detailed review answers: [[Media:IoTSec-ReviewerComments_247627.pdf]] |
Latest revision as of 18:19, 17 August 2015
Security in IoT for Smart Grids | |||||||
---|---|---|---|---|---|---|---|
|
![]() |
Comments from Reviewer's to the proposal
The decisions regarding approval and rejection of grant proposals have been taken by the Komite for IKTPLUSS bevilgende utvalg. The comments of the referee panel were used to assist in the decision-making process. They are being forwarded here to provide you with scientific feedback, and do not comprise the grounds for the funding decision.
This project addresses a timely and challenging topic. The team has a very good background in the area and builds on existing collaborations, bringing together academia and industry. The scientific expertise is very good but not excellent.
The project lacks details regarding how research results will be integrated, and how industrial partners will make use of project results. The dissemination plan is very solid as it targets typical publication activities but also organization of specific Workshops, approach to standardization bodies and approach to European Technology Platforms.
- Comments: The main integration will take place along two lines, being (i) the simulation of security models and modules, and (ii) the integration in the test-bed by NR and the showroom of NCE Smart and eSmart Systems. Task T4.1 is updated to address the industrial ecosystem for security in IoT, and thus build the basis for a common understanding of research and industry.
- Comments: Industrial partner will make use of the results through the adaptation for the security centre for Smart Grids, addressed in WP4.
The overall plan could be improved by being more selective and focusing on quality instead on quantity and wide coverage. Several challenges remain without a convincing answer.
- Comments: Thanks for the comment on quality, rather than quantity and wide coverage. Our definition of quality follows the academic standards, being (i) the description in a technical note, (ii) the presentation on a peer-reviewed conference, and finally (iii) the peer-reviewed journal paper. We will first prepare the ground in a wide sense, and then use the feedback from the industrial partners and their requirements for the Security Centre to focus on specific challenges. Thus, we expect that we can provide a more focussed approach at M06 (after the second progress meeting) with a more streamlined approach influenced by Industry.
Prosjektet vurderes til et "meget godt" forskerprosjekt.
Assessment of relevance:
Prosjektets relevans i forhold til utlysningen vurderes å være "meget godt".
Prosjektet bidrar i stor grad til utvikling av sterke og robuste nasjonale forskningsgrupper. Prosjektet involverer et FoU-samarbeid mellom flere sentrale forskningsmiljø. I tillegg er flere energiselskaper med som brukerpartnere. Prosjektet har målsetting om å bidra til en nasjonal klynge innen fagfeltet og har derfor klare kompetansebyggende ambisjoner. Utdanning av tre phd stipendiater og finansiering av tre post doc stipender vil bidra til dette.
Prosjektet bidrar noe til å styrke forskningsmiljøenes internasjonale konkurransekraft. Det skyldes i hovedsak et allerede et godt internasjonalt nettverk blant partnerne. Hvilke nye internasjonaliseringsaktiviteter prosjektet skal bidra med sies det lite om og det er heller ikke dokumentert at det inngår noe internasjonalt samarbeid.
- Comments: Here we have to be more precise, on how to build the international network. It includes a cross-European network for IoTSec, with e.g. a prototype network at Mondragon University (MGEP), IoTSec models with Viktoria University, an IoTSec course with University Tor Vergata, and the exchange of key personnel both ways, ... Regarding the European Technology platforms and collaborations, DG14 Energy invited us to the ICT 2015 conference to join a Network Session on Data Management, initiated by our collaboration project H2020 Empower.
Prosjektets bidrag til utvikling av nye innovasjoner og samfunnsnytte forventes å være betydelig. Innovasjonspotensialet for sikre IoT løsninger er stort og tett involvering og utstrakt samarbeid med brukermiljøene i et slikt prosjekt vil kunne sikre at forskningsresultatene raskere finner anvendelser til nytte for samfunnet.
Detailed Review
Scientific Merits
The project considers system description, security modelling, evaluation and industrial applications in the context of secure applications for smart grids scenarios using IoT technologies. The research proposed will produce new techniques, mechanisms and methodologies, but the proposal lacks details regarding how those aforementioned results will be integrated. Overall, the focus seems to be on more applied research that is relevant to industry. The project will complement an existing collaboration with a cluster of projects. The approach of "semantic provability" is promising. The a priori knowledge of the project about the state-of-the-art is well documented and the scientific basis is realistic.
Project manager and project group
The consortium is composed of a good combination of research and industrial partners. Actually, they conform a very representative group for the particular scope of the proposal, but with limited presence in the top scientific security venues.
- Comment: Through the aforementioned approach of scientific quality we will address top scientific venues.
The project manager is active, productive and has experience from several national and EU projects with strong industrial background, but his track record in security, privacy and dependability is rather weak. Overall his citation profile could have been stronger.
- Comment: The project leader was leader of the JU Artemis pSHIELD project, and industrial applicability leader of the JU Artemis nSHIELD project. Through the projects the methodology of measurable security, privacy and dependability was established. Project leadership and industrial applicability had a higher priority than academic publications.
International collaboration within the scope of the project is foreseen due to previous and current involvement of project partners in research relationships, at different levels (projects, publications, ...) with European universities and companies
Implementation plan and resources
The project has very detailed workpackages, tasks and milestones, showing an overall clear structure. The roles and responsibilities of the partners are well-defined. Also, resources requested are reasonable considering the size of the consortium and duration of the project. However, the implementation plan lacks to clearly show how industrial partners will make use of project results.
The quantity and diversity of approaches produces some concern about the focus and integration.
- Comment: The quantity of approaches was introduced to define the basis on top of which IoTSec will define the focus. The focus of the project, applicability in a Smart Grid Security Centre, is highlighted through an updated task description in WP4. A detailed task and workpackage description will address which approaches are taken into consideration.
It is not clear that there is a validation planned with real users, even if usability seems to be one of the goals.
- Comment: Validation with real users is addressed in two ways, being (i) the simulations in the eSmart show room and (ii) the applicability in the Smart Grid Security Centre.
Overall, it could have been made more clear what the research outcomes will be beyond research papers and project proposals.
- Comment: The research outcomes are better addressed through the extended descriptions of the tasks and the workpackages, including detailed objectives and expected outcome.
Dissemination and communication of results
The dissemination plan is very solid, broad and detailed as it targets, in a realistic way, not only typical publication activities but also organization of specific Workshops, approach to European Technology Platforms, and approach to standardization bodies, which is essential in this area. It should also involve the top security conferences that guarantee the highest visibility in the international community. The proposal has measurable outcomes in terms of dissemination, what is valuable.
Overall assessment
This project addresses a timely and challenging topic. The team has a very good background in the area and builds on existing collaborations, bringing together academia and industry. The scientific expertise is very good but not excellent. The project lacks details regarding how research results will be integrated, and how industrial partners will make use of project results. The dissemination plan is very solid as it targets typical publication activities but also organization of specific Workshops, approach to standardization bodies and approach to European Technology Platforms. The overall plan could be improved by being more selective and focusing on quality instead on quantity and wide coverage. Several challenges remain without a convincing answer.
all taken from the detailed review answers: Media:IoTSec-ReviewerComments_247627.pdf