congrats.
Erica Breslau: hi everybody i'm not sure if this actually started or not, but I want to welcome you to module five.
Erica Breslau: And to our speakers beth Glenn who was from UCLA and Maria Fernandez who's from the University of Texas.
Erica Breslau: A couple of housekeeping first of all, I just want to congratulate you on using the modules to post your questions we've had a really dynamic back and forward with questions.
Erica Breslau: You do have a preference to post them either under the small group module which is probably preferred, but if you have a question for the entire multi cohort please continue to use the ledger module we will check these throughout the week.
Erica Breslau: And if you can get your questions in by Tuesday, then we can get them to our speakers.
Erica Breslau: And with that i'm happy to hand this over to Brian who will moderate the questions today.
Brian Mittman: Great Thank you and my thanks as well to our lectures and special guests for the Q amp a will have about 30 minutes of discussion and Marie Fernandez will join some of the small groups that Glenn has a conflict and cannot so.
Brian Mittman: You know, please continue to post questions through chat so unreal first question goes to you, and that is a very.
Brian Mittman: very narrow specific question, and that is whether there are ever areas or circumstances under which intervention mapping would not be appropriate, or perhaps that might be useful, but cannot be used, and if so.
Brian Mittman: What would you suggest.
Maria Fernandez_ Texas Prevention Research Center: yeah, thank you for that question and you know, I think that it's rare when I went and when I would say no, it can't be used at all, I think that the question of you know.
Maria Fernandez_ Texas Prevention Research Center: Can you use it from sort of start to finish, is is that it depends, it depends on where you are in the planning process have you already selected strategy and.
Maria Fernandez_ Texas Prevention Research Center: Now you're just kind of trying to build them out or tailor them in some way.
Maria Fernandez_ Texas Prevention Research Center: Or are you really starting from the needs and assets assessment where you're you're exploring and you're really open any kind of strategy.
Maria Fernandez_ Texas Prevention Research Center: And I think that, depending on where you are in the process.
Maria Fernandez_ Texas Prevention Research Center: You would you would you know do use it fully use the process fully from start to finish, or you might just kind of jump in and middle and it's it's really been interesting.
Maria Fernandez_ Texas Prevention Research Center: You may know that there is going to be a special topic issue on implementation mapping right so intervention mapping.
Maria Fernandez_ Texas Prevention Research Center: Is a whole process for planning multi level interventions and implementation mapping is a process for planning, implementation strategies, whether they're multi level or not.
Maria Fernandez_ Texas Prevention Research Center: And, but we have a special topics issue coming up in printers and public health, and I was i've been looking at some of the.
Maria Fernandez_ Texas Prevention Research Center: Some of the papers it's been really interesting to see the various ways that people are are plugging into the process right and how the process is helping them.
Maria Fernandez_ Texas Prevention Research Center: depending on where they are in their planning of implementation, strategies and I think the same is true in in planning multi level intervention, so I would say that it would be rare.
Maria Fernandez_ Texas Prevention Research Center: The instance in which it wouldn't be helpful at all, but I think that, depending on where you are in the process.
Maria Fernandez_ Texas Prevention Research Center: You you might decide to use all of it, some of it just started like this intervention mapping light, I think, which is i'm not really going to do it as prescribed but i'm still going to use the logic of it as i'm making decisions for various.
Maria Fernandez_ Texas Prevention Research Center: components of the Multi level intervention so i'm happy to go deeper into that but.
Maria Fernandez_ Texas Prevention Research Center: that's helpful best.
Brian Mittman: Let me ask the follow up.
Brian Mittman: You know, those of us who have some experience using intervention that and realize that covers approaches that have been.
Brian Mittman: passed on to other nations and, in some ways it sort of summarizes practice them but I bet you can sort of imagine yourself in a scenario where you have no familiarity at all with intervention mapping inner lining either on all your approaches or.
Maria Fernandez_ Texas Prevention Research Center: You know.
Brian Mittman: intuition, what are some of the other methods that are available to go about Designing an intervention and based on some form of the needs assessment that may even include the more of an informal intuitive.
Brian Mittman: understanding of the very person needs.
Beth Glenn: He said to me right, Brian.
Brian Mittman: Yes, that's yes.
Beth Glenn: yeah um I would say that um you know I think you know the different ways of preparing and you know doing your formative research, I mean I there's different kinds of paths to get to get there, but I think that.
Beth Glenn: You know some of the elements of intervention mapping are at play and kind of a more you know kind of traditional approach to developing.
Beth Glenn: An intervention, so I think you're going to you know if you have your own formative work looking at you know factors that are associated with your outcome at you know your multiple levels.
Beth Glenn: But you also are going to need to look at you know the literature and understand kind of what have.
Beth Glenn: what's been found in the past and in the lecture that I gave about the HPV work we're doing.
Beth Glenn: You know, in much of the work we do focused on parents and in HPV at least parents and other.
Beth Glenn: other kind of areas of health, we would really focus on making sure parents understand how important vaccines are making sure they're.
Beth Glenn: They know about you know, the low risk level things like that, like that is really how.
Beth Glenn: we've always designed interventions at the individual level in this case, it was apparent level, but we also have.
Beth Glenn: literature right that's showing that a simple announcement approach is more effective and so, in that case we kind of.
Beth Glenn: You know bypass what some of the previous literature and other areas has has told us and and utilize kind of what we found from previous studies, so I think it's that combination, but definitely thinking about you know what are the leavers.
Beth Glenn: That that you believe, are you know under lying the behavior you want to change and and figuring out kind of what our potential options or.
Beth Glenn: or moving those leavers and you know some of that's going to come from, of course, you should come from you know what theoretically, do you think is important, but then also is there literature, we can draw on.
Beth Glenn: Do you have personal experience, is there any anecdotal you know experience of the clinic or the system within which are working, that come into play, and I think it's putting those pieces together.
Brian Mittman: So let me follow up and ask question in Korea.
Brian Mittman: opposed to you first.
Brian Mittman: That has to do with issues feasibility and I don't know comfort, you know what i'm about to ask is a version of you know, the.
Brian Mittman: story of the drunk looking for his keys under the lamppost, even though there were no lost at the other end of the alley in the dark.
Brian Mittman: So, so when we think about designing and eventually deploying and evaluating an intervention there are issues related to our comfort.
Brian Mittman: Our areas of expertise and familiarity the degree of literature that exists and that again we are comfortable with.
Brian Mittman: The level of the literature, rather, but also when you know, the issue of our degrees of levels of influence, so no bottom line is, in many instances we gravitate to the.
Brian Mittman: Patient client level to focus our efforts there you know, we have the background in health psychology we have the ability to.
Brian Mittman: You know deploy interventions at that level where it's you know all the way at the top of the onion policy level, you know, clearly, we have no ability to influence federal regulations so.
Brian Mittman: Apologies for the long winded way of posing the question, but how do we go about balancing those considerations and in what are some practical methods that we can use to ensure that we at least have.
Brian Mittman: You know, an understanding of all the levels that ideally would be addressed before we then you know narrow to those that we have some you know ability to address a Maria your thoughts.
Maria Fernandez_ Texas Prevention Research Center: So i'm and i'm gonna i'm gonna link in a follow up to the previous question, with an answer to your question so.
Maria Fernandez_ Texas Prevention Research Center: You know a lot of times when when people think about intervention mapping.
Maria Fernandez_ Texas Prevention Research Center: gosh you don't have to learn this whole new process and in detail, and even if you look it up on Wikipedia says something like useful but very complicated right and.
Maria Fernandez_ Texas Prevention Research Center: And I you know I like to tell people that it's there for whatever we want to use it right and so there's some.
Maria Fernandez_ Texas Prevention Research Center: Fundamental philosophies and in fact we call them the core processes of intervention mapping that basically say that at every step along the way, including the question Brian and just ask and i'll get to in a minute.
Maria Fernandez_ Texas Prevention Research Center: And every step along the way you have to make decisions and, and one way to make decisions is to think about how to prioritize.
Maria Fernandez_ Texas Prevention Research Center: And, and so you know part of the core processes of.
Maria Fernandez_ Texas Prevention Research Center: Intervention happy when you describe them is looking some I think that's it but you're not fearing looking at evidence collecting data engaging communities right So these are sort of.
Maria Fernandez_ Texas Prevention Research Center: Some of the things that in your toolbox, if you will, to help make decisions along the way.
Maria Fernandez_ Texas Prevention Research Center: And then prioritization is a really important piece of that and and prioritization no matter if you're thinking about what priority determinants Do I need to.
Maria Fernandez_ Texas Prevention Research Center: emphasize here to change this behavior or what are the priority condition at various levels in the environment that I want to focus on we think about.
Maria Fernandez_ Texas Prevention Research Center: Important so how important is this in terms of the overall outcomes and changing though changeability or feasible.
Maria Fernandez_ Texas Prevention Research Center: Now this is this is getting to your question, Brian where people often the reason why people often gravitate towards focusing on patient.
Maria Fernandez_ Texas Prevention Research Center: or Community members and you're focusing on providing a better because they feel like well that's way more changing something else policy.
Maria Fernandez_ Texas Prevention Research Center: The policy environment may be more powerful in terms of change, but it's just not feasible.
Maria Fernandez_ Texas Prevention Research Center: And what we try to do when we have when we sort of try to nudge people to think a little bit more deeply and to if there is a possibility to make a change.
Maria Fernandez_ Texas Prevention Research Center: Is in those higher order levels is you know sneak beyond your own training, when we talk about to science multidisciplinary approaches, so you may need to include a policy experts or others if, in fact, the greatest leavers.
Maria Fernandez_ Texas Prevention Research Center: Are those that are on higher levels, so I would say one super important to capture it document that it's important, even if you think it's beyond the scope of your particular program and and second i'm lean into the discomfort of trying to.
Maria Fernandez_ Texas Prevention Research Center: consider creating strategies to make change at those higher levels, even if it means you know, including other people beyond the team, I think that knowing look at public health impact.
Maria Fernandez_ Texas Prevention Research Center: Some of the greatest change has been when you made change that goes higher order so it may not be possible, consider importance and visibility, but but lean into trying to to include that as much as possible okay sounds.
Maria Fernandez_ Texas Prevention Research Center: Good.
Brian Mittman: Thanks beth other thoughts.
Beth Glenn: I agree with everything Maria said, I think you know, we also have the restriction of the of the five year you know our a one cycle, and you know just kind of the the fact that policy changes typically doesn't follow our.
Beth Glenn: You know our intervention period timeline and from our grant, but I think that yeah, I think, really, really important that.
Beth Glenn: That you kind of figure out what is the highest young kind of sphere of influence that you think you can make a change in, and then I think.
Beth Glenn: Meaning what you're all here for an implementation science training, so you know, but how important it is to document, even the levels that you don't think you can necessarily change and.
Beth Glenn: or even the ones that are going to be tricky to change because you know, in the end of the day, if you don't see an effect, then it's just really important to be able to say we really think this is the most important lever unfortunately in this.
Beth Glenn: project we didn't move it and you know no wonder, we didn't have the effect we we thought we might, so I think you know kind of the documenting making sure.
Beth Glenn: you're kind of you know, like looking at all the levels and documenting what you're doing and I think the collaboration.
Beth Glenn: is really important, but also sometimes we have to have the added issue of like.
Beth Glenn: Is the policy at a level that's so high that there's only one policy environment in you know, and you can't manipulate that so, so I think that sometimes measuring is the best we can do, but I think.
Beth Glenn: You know kind of like like reassigned going out of your comfort zone and and really documenting kind of what is taking place, and how your intervention may or may not be changing it is super important.
Maria Fernandez_ Texas Prevention Research Center: For example, a good example, I think, is that the catch program the coordinated approach to child health, which is a multi level intervention and.
Maria Fernandez_ Texas Prevention Research Center: It was developed at ut health and then you know disseminated in various schools around around the state, but perhaps one of the biggest impacts in terms of getting us was when it became.
Maria Fernandez_ Texas Prevention Research Center: You know, a requirement, so when to the Texas legislature, the Texas legislature, made the decision that all schools had to have a program like this right, I mean they didn't mention catches.
Maria Fernandez_ Texas Prevention Research Center: exclusively but they had to have a program like this and and and looking at well how did that happen, who had to do what, in order to make that happen.
Maria Fernandez_ Texas Prevention Research Center: And and thinking and so looking at some of those examples I think that that can be helpful, but, but again to best point.
Maria Fernandez_ Texas Prevention Research Center: There is lots of different levels of policy, sometimes we think of policy and our mind automatically goes to universal health care coverage and you're going to have that right and and but but I mean there's lots of different levels policy that that may be worn changing.
Brian Mittman: So let me go in a slightly different direction, because there were a couple of questions that came in having to do with complexity, over time, and the fact that for some kinds of problems that we're hoping to address.
Brian Mittman: You know, there are a number of levels, a number of target processes that need to be changed and it's logical to think of doing this in a sequential manner.
Brian Mittman: There are also situations where the needs change over time and one of the examples given with some cancer survivorship where the early stage activities in needs are different from those you know that occur, later on, how do we deal with that complexity.
Brian Mittman: You know, one of the trainees asked about developing separate individual intervention maps and then.
Brian Mittman: You know, having an overarching framework that.
Brian Mittman: guides the way that you use them over time, so you know different different forms of complexity and evolution in changing needs but beth your thoughts on.
Brian Mittman: How to handle that complexity and variability.
Beth Glenn: yeah that's that's a tricky tricky one without this more a little bit more specific context I think um, but I think that's also one of the challenges of this kind of work is just that.
Beth Glenn: We see you know we're developing protocols and you know kind of guidelines for what we want our interventions to look like, but then also we know.
Beth Glenn: That we want to kind of see what's working and what isn't it we don't want to just say let's wait till five years and see whether or not the intervention work, so I think that.
Beth Glenn: We have that ability, even though it makes things trickier to you know to to make modifications and try to you know kind of do what we can to not just wait and see, but to actually.
Beth Glenn: You know kind of increase the intensity as we need or to make changes, when we see that things don't seem feasible or don't seem to be working.
Brian Mittman: What about let me draw an example from project that we're working on together, and that is.
Brian Mittman: So we think about variability and heterogeneity let's focus first on heterogeneity across place so in our Kaiser you know region southern California.
Brian Mittman: We know that the needs differ across you know different sites in different parts of the region so.
Brian Mittman: Is that a situation where separate done intervention maps and be appropriate for the individual clinics, or is there a way of developing, one that has the flexibility to account for the different needs and barriers that we see.
Beth Glenn: And I think that's a good question for Maria, but I think that, like like I know the work you've been doing Ryan kind of thinking about like what's the general intervention approach and then, how does it get implemented and there's kind of room for both.
Beth Glenn: doesn't really want to add to that.
Maria Fernandez_ Texas Prevention Research Center: yeah so a couple things, first let me take this I think there's to me, at least on organizing this in my mind right now.
Maria Fernandez_ Texas Prevention Research Center: And one has to do with the fact that intimate.
Maria Fernandez_ Texas Prevention Research Center: moments across at different levels and Brian you first asked the question you were talking about timing and I think that.
Maria Fernandez_ Texas Prevention Research Center: The answer is yes, you do have your timing right what the most immediate one that comes to mind right now it's not a cancer example it's a.
Maria Fernandez_ Texas Prevention Research Center: stroke example when, and this is going to Dayton, this was a long time ago when when TPA right that the clot busting drug for became available.
Maria Fernandez_ Texas Prevention Research Center: And we are working with these rural communities to to get it more available so so.
Maria Fernandez_ Texas Prevention Research Center: i'm providers and those emergency rooms, the emergency room service personnel and the protocols, but also the Community, so that people would would go to the er and also their primary care Doc so there's a lot of different.
Maria Fernandez_ Texas Prevention Research Center: levels and people involved right to me thought about the different components of this multi level intervention, it was really clear that the capacity at the er how that happened first.
Maria Fernandez_ Texas Prevention Research Center: Like that we didn't want to be driving people to the er if then we weren't going to be getting what they needed to be getting right so.
Maria Fernandez_ Texas Prevention Research Center: Sometimes, as you as you look at all of the various relationships, and this is where I think like a system science association really useful because you see like different arrows and how this affects this matter of fact that.
Maria Fernandez_ Texas Prevention Research Center: You know another thing that was happening in that same example is that people would call their primary care physician and the primary care physician and say.
Maria Fernandez_ Texas Prevention Research Center: You know call me in the morning and whatever and and so, so there were there were some work that needs to be done there even before.
Maria Fernandez_ Texas Prevention Research Center: Some of the other pieces, so, in any case mapping it out seeing what influences what I think can be really helpful in terms of the timing of those of those various levels, and how you implement those various levels.
Maria Fernandez_ Texas Prevention Research Center: The other thing that you asked about the variability encouraging know how we talked about defining or dissemination.
Maria Fernandez_ Texas Prevention Research Center: right that you wanted to divine things so that they're equally valuable it is a you know, easy to implement right from the start, and you can do the same thing with adaptation you designed for adaptability.
Maria Fernandez_ Texas Prevention Research Center: And, and one way that you might do, that is, as you understand and and and map out some of the differences you're also going to know what's not different.
Maria Fernandez_ Texas Prevention Research Center: And, and so there may be like a core of your multi level intervention and then there's some variations depending on different ways that it might be deliver different people that might be involved, and you can think about and plan for that right.
Brian Mittman: And that's to put a plug in that's a situation where I would of course advocate for our thinking about core functions and forums and.
Brian Mittman: Ensuring that you have a consistent set of core functions, but the forums that menu or forums would be available, let me.
Brian Mittman: Excuse me, let me jump to a much more practical question that you know it's likely on the minds of all of us, and that is how do we deal with.
Brian Mittman: The constraints of an r1 and the need to know approach our intervention design in a more logical thoughtful manner, you know we all tend to in the past, it was almost.
Brian Mittman: Dominant approach to describe the intervention with perhaps you know some basis and pilot data and literature review in the er one begins with the intervention deployment is it.
Brian Mittman: advisable to include the needs assessment in the intervention process mapping process and the intervention evaluation in the single are one.
Brian Mittman: Or would you split across the perhaps in our three or 21 for some of the preliminary work really what has your experience been both in terms of what you find to be most feasible, but also what you find to be most fun the ball.
Maria Fernandez_ Texas Prevention Research Center: Now, part of the answer to this well, let me just say right off the BAT done it both ways right so done both ways were some of the.
Maria Fernandez_ Texas Prevention Research Center: original intervention development work and planning work has been in the previous section are three or one.
Maria Fernandez_ Texas Prevention Research Center: That i've also done where everything happens in our like usually and it may be the first game is really.
Maria Fernandez_ Texas Prevention Research Center: informative valuation work testing, etc, and then, then it goes into trial.
Maria Fernandez_ Texas Prevention Research Center: But within I would say that second approach you still do have to go in, obviously with some preliminary data, so you might not have your intervention fully.
Maria Fernandez_ Texas Prevention Research Center: developed, but you have some some evidence that different pieces of it, or that the right your timing right determinant or do some some other pieces you still have to have some.
Maria Fernandez_ Texas Prevention Research Center: Data but for sure i've gotten a grant where your it's it's part of that are on the development, these are one Now I know you know.
Maria Fernandez_ Texas Prevention Research Center: People and probably talked about this to you before and maybe you've experienced it yourself, one of the tricky things when you're writing grants.
Maria Fernandez_ Texas Prevention Research Center: particularly one that expects you to have Community engagement and development and refinement process is expected to be part of the application.
Maria Fernandez_ Texas Prevention Research Center: If it sometimes sounds like we're contradicting ourselves right because we're saying we're going to do this.
Maria Fernandez_ Texas Prevention Research Center: And then we're saying we're going to engage the Community and do the work to see what we're going to do.
Maria Fernandez_ Texas Prevention Research Center: Right and and all I can say about that is that it's something that we all struggle with and it's a bit of a grandson ship anything where you, you were clearly describing the process, such that.
Maria Fernandez_ Texas Prevention Research Center: The reviewer understands that you're you're being careful about considering theory and evidence and data cetera.
Maria Fernandez_ Texas Prevention Research Center: And at the same time you're you're telling them, and this is what we think it may look like, because otherwise.
Maria Fernandez_ Texas Prevention Research Center: You will get the comment that you know we don't know what you're doing we don't know what it looks like and and I in it again it's frustrating and it's a struggle, but you kind of have to play, both sides, a little bit that's that's my opinion in my experience, but happy to hear.
Maria Fernandez_ Texas Prevention Research Center: That.
Brian Mittman: that's your one of those others your experience and your recommendations and advice.
Beth Glenn: yeah no I think i'm Maria really hit the nail on the head.
Beth Glenn: it's this, I do think you know it's really important to have the preliminary data, and I think that there's been like a lot of creep over time, with the expectations for preliminary data so that's you know, the challenge that you're all facing.
Beth Glenn: But yeah we often have kind of like a finalize the intervention aim one.
Beth Glenn: So, meaning you're showing them that you've thought about it, you have some preliminary data, but yet you have that ability to tweak it and you're going to you know do it in collaboration with you know, whoever you're partnering with and.
Beth Glenn: To make those kind of final adjustments and not to mention the fact that you're trying to use this you know implementation science framework of.
Beth Glenn: You know not not heavily prescribing every little element of the intervention, but I do think it is, is a push and pull in that yes, and we've.
Beth Glenn: You know, had to write entire grants where you're talking about Community collaboration and you know you got the rfa like 60 days ago and.
Beth Glenn: you're you know you're writing what you're going to do, because otherwise you won't get the grant but, yet you say you're going to do it with the Community, so I think that yeah it is.
Beth Glenn: Just accepting that that is one of the challenges and and doing your best really is the is the most competitive that you can be, I think.
Brian Mittman: So we have time for one more question and beth i'd like you to answer first, because I know you need to run to another meeting in it, you know you both addressed it in part.
Brian Mittman: But the specific question has to do with the level of evidence in the prior preparatory work on the individual single level intervention so molecular level intervention puts together how many parts.
Brian Mittman: Do we need to wait until we have good hard evidence for each of the individual signal levels before we can propose.
Brian Mittman: To put them all together, or is it possible to include some new elements at some the levels, combining with existing established elements that others and hope that reviewers will.
Brian Mittman: score it sufficiently to be funded.
Beth Glenn: yeah I would say that, yes, it's definitely possible and I would say we shouldn't wait.
Beth Glenn: till we have everything perfect because the reviewers still won't see it that way, so that's just the reality, so I think that.
Beth Glenn: I think it's good to have you know really good evidence for a level or two.
Beth Glenn: But then, you know, I think, adding additional elements it even if sometimes necessary it's what we would expect it's how science moves forward, so I think it is okay um and I think you don't want to wait.
Beth Glenn: Again, to feel like you're perfectly prepared, because we know you know the timeline you're going to have to go through the review process more than one time.
Beth Glenn: And you know also sometimes if you're say like well i'm going to start doing that provider element, you know next month, but I couldn't submit it in two months, but I won't have that much data, I mean.
Beth Glenn: Just submitted and then you know, the fact is you'll have that provider data, the next time you need to reach resubmit so sometimes that's actually really helpful too, because you, you seem really responsive, but you didn't wait too long and delay your you know kind of your timeline.
Brian Mittman: Great thanks, and thanks again for joining us this morning for us and California Maria your thoughts about the, the issue of how much evidence is needed, and how much new.
Brian Mittman: development work can be performed on individual components of the intervention.
Maria Fernandez_ Texas Prevention Research Center: I think i'm.
Maria Fernandez_ Texas Prevention Research Center: Just like the comment that I sent you want some preliminary preliminary studies on area data.
Maria Fernandez_ Texas Prevention Research Center: When you're doing an Roi I think that that's true regardless of the level right, so you, ideally, you would want some preliminary.
Maria Fernandez_ Texas Prevention Research Center: Data on on what you're doing each of the levels, or at least what factors are influencing the outcomes at those levels right.
Maria Fernandez_ Texas Prevention Research Center: And having said that, I think that I agree with that um you shouldn't wait at that it's perfectly fine to be testing multiple.
Maria Fernandez_ Texas Prevention Research Center: Levels of testing components at multiple levels, the tricky thing is, though, major event it matches your research question so, what are your primary research questions so sometimes you're.
Maria Fernandez_ Texas Prevention Research Center: In and where you towering So how are you powering study are retiring at at the station level on.
Maria Fernandez_ Texas Prevention Research Center: The provider or the patient, I think that that matters, and so you all are familiar with a hybrid designs right and and sort of where the emphasis in.
Maria Fernandez_ Texas Prevention Research Center: meditation science, whether it's on your communication strategy around intervention.
Maria Fernandez_ Texas Prevention Research Center: By the same token, when you're talking about a multi level intervention, you may want to think through where's the emphasis where Am I what am I really.
Maria Fernandez_ Texas Prevention Research Center: trying to get the answer for me ultimately it's a package right it's across all levels in terms of the outcome that's that's why you're doing multi level but, but you still have to make some decisions later where you're doing the assessments how empowering etc.
Brian Mittman: Okay, I think, in the interest of time, but we need to wrap up even though we've addressed only about a quarter of fewer of the total questions, we will see how many.
Brian Mittman: We could address and writing and provide them via email and through confluence but.
Brian Mittman: I think we are ready to split into our separate groups so again, Maria my thanks to you as well, for your lecture and joining us and for staying through during some of the small groups Eric anything else that you need to say before we begin to what.
Erica Breslau: No, I think that this was a very rich discussion, and I think some of the questions may come up during the small groups, I certainly hope that some of the questions that were asked under individual small groups will be addressed.
Erica Breslau: So we look forward to seeing you in a few minutes.
Erica Breslau: Yes, Maria you're going to go to group.
Erica Breslau: I think it's group three.
Erica Breslau: But you're going to be put into the group.
Erica Breslau: needed, could you.
Erica Breslau: split everybody into this small groups, please.
Anita Peterson: Yes, I will please stand by.